[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:37:50 dist-test-slave-wwwb krb5kdc[22588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:37:50 dist-test-slave-wwwb krb5kdc[22588](info): set up 2 sockets
Sep 28 14:37:50 dist-test-slave-wwwb krb5kdc[22588](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260928 14:37:51.490068 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:37:51 dist-test-slave-wwwb krb5kdc[22588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:37:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:37:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:37:51.612919 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:43531
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:43339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:43531
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:37:51.694299 22601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:37:51.694561 22601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:37:51.694597 22601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:37:51.695813 22601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:37:51.695885 22601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:37:51.695909 22601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:37:51.695941 22601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:37:51.695966 22601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:37:51.697484 22601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:43339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:43531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:43531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:37:51.697783 22601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:37:51.698076 22601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:37:51.703636 22607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:51.703681 22601 server_base.cc:1034] running on GCE node
W20260928 14:37:51.703639 22606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:51.703655 22609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:51.704209 22601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:37:51.704489 22601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:37:51.705636 22601 hybrid_clock.cc:648] HybridClock initialized: now 1790606271705622 us; error 32 us; skew 500 ppm
Sep 28 14:37:51 dist-test-slave-wwwb krb5kdc[22588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606271, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:37:51.708256 22601 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:37:51.708729 22601 webserver.cc:460] Webserver started at http://127.22.13.126:44023/ using document root <none> and password file <none>
I20260928 14:37:51.708961 22601 fs_manager.cc:359] Metadata directory not provided
I20260928 14:37:51.709009 22601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:37:51.709169 22601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:37:51.710428 22601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "8647ddf87d9545669080378232068c19"
format_stamp: "Formatted at 2026-09-28 14:37:51 on dist-test-slave-wwwb"
server_key: "dcc77f10362bbbdecdd431d8019bf86d"
server_key_iv: "0b91612a3b17509dae1ba698cc36ade2"
server_key_version: "encryptionkey@0"
I20260928 14:37:51.710809 22601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "8647ddf87d9545669080378232068c19"
format_stamp: "Formatted at 2026-09-28 14:37:51 on dist-test-slave-wwwb"
server_key: "dcc77f10362bbbdecdd431d8019bf86d"
server_key_iv: "0b91612a3b17509dae1ba698cc36ade2"
server_key_version: "encryptionkey@0"
I20260928 14:37:51.712212 22601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:37:51.712952 22616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:51.713137 22601 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:37:51.713233 22601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "8647ddf87d9545669080378232068c19"
format_stamp: "Formatted at 2026-09-28 14:37:51 on dist-test-slave-wwwb"
server_key: "dcc77f10362bbbdecdd431d8019bf86d"
server_key_iv: "0b91612a3b17509dae1ba698cc36ade2"
server_key_version: "encryptionkey@0"
I20260928 14:37:51.713304 22601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:37:51.745306 22601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:37:51.747323 22601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:37:51.747481 22601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:37:51.751747 22601 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:43531
I20260928 14:37:51.751816 22668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:43531 every 8 connection(s)
I20260928 14:37:51.752185 22601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:37:51.753072 22669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:51.755473 22669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19: Bootstrap starting.
I20260928 14:37:51.756119 22669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:51.756438 22669 log.cc:824] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19: Log is configured to *not* fsync() on all Append() calls
I20260928 14:37:51.757174 22669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19: No bootstrap required, opened a new log
I20260928 14:37:51.758679 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 22601
I20260928 14:37:51.758764 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:37:51.758898 22669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8647ddf87d9545669080378232068c19" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43531 } }
I20260928 14:37:51.759061 22669 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:51.759109 22669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8647ddf87d9545669080378232068c19, State: Initialized, Role: FOLLOWER
I20260928 14:37:51.759258 22669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8647ddf87d9545669080378232068c19" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43531 } }
I20260928 14:37:51.759334 22669 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:37:51.759377 22669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:37:51.759429 22669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:51.759517 22581 external_mini_cluster.cc:1428] Setting key f6ed553a1c0191f4e7fe1bf22bb1d247
I20260928 14:37:51.760046 22669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8647ddf87d9545669080378232068c19" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43531 } }
I20260928 14:37:51.760164 22669 leader_election.cc:302] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8647ddf87d9545669080378232068c19; no voters:
I20260928 14:37:51.760350 22669 leader_election.cc:288] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:37:51.760458 22672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:37:51.760618 22669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:37:51.760656 22672 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [term 1 LEADER]: Becoming Leader. State: Replica: 8647ddf87d9545669080378232068c19, State: Running, Role: LEADER
I20260928 14:37:51.760846 22672 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8647ddf87d9545669080378232068c19" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43531 } }
I20260928 14:37:51.761399 22674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8647ddf87d9545669080378232068c19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8647ddf87d9545669080378232068c19" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43531 } } }
I20260928 14:37:51.761487 22674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [sys.catalog]: This master's current role is: LEADER
I20260928 14:37:51.761636 22675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8647ddf87d9545669080378232068c19. Latest consensus state: current_term: 1 leader_uuid: "8647ddf87d9545669080378232068c19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8647ddf87d9545669080378232068c19" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43531 } } }
I20260928 14:37:51.761713 22675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19 [sys.catalog]: This master's current role is: LEADER
I20260928 14:37:51.761840 22683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:37:51.762454 22683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:37:51.768631 22683 catalog_manager.cc:1347] Generated new cluster ID: f6c7523c6da94edbacedac8351a4eacb
I20260928 14:37:51.768705 22683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:37:51 dist-test-slave-wwwb krb5kdc[22588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:51.779846 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:51.760309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45030 (local address 127.22.13.126:43531)
0928 14:37:51.762306 (+ 1997us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:51.762317 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:51.762732 (+ 415us) server_negotiation.cc:407] Connection header received
0928 14:37:51.765481 (+ 2749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:51.765489 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:51.765547 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:51.765687 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:51.768664 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:51.769718 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:51.771514 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:51.772169 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:51.772266 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:51.776022 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:51.776026 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:51.776027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:51.776039 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:51.777788 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:51.778282 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:51.778284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:51.778285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:51.778372 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:51.778638 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:51.778639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:51.778639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:51.778893 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:51.779062 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:51.779341 (+ 279us) server_negotiation.cc:299] Negotiation successful
0928 14:37:51.779478 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1948,"thread_start_us":72,"threads_started":1}
I20260928 14:37:51.780745 22683 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:37:51.781445 22683 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:37:51.787992 22683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8647ddf87d9545669080378232068c19: Generated new TSK 0
I20260928 14:37:51.788219 22683 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:37:51.818499 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:43531
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:43339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:37:51.914508 22695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:37:51.914731 22695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:37:51.914758 22695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:37:51.916057 22695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:37:51.916101 22695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:37:51.916132 22695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:37:51.917737 22695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:43339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:43531
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:37:51.918033 22695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:37:51.918294 22695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:37:51.923377 22701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:51.923400 22700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:51.923516 22703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:51.923728 22695 server_base.cc:1034] running on GCE node
I20260928 14:37:51.923935 22695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:37:51.924183 22695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:37:51.925340 22695 hybrid_clock.cc:648] HybridClock initialized: now 1790606271925323 us; error 31 us; skew 500 ppm
Sep 28 14:37:51 dist-test-slave-wwwb krb5kdc[22588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606271, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:37:51.928408 22695 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:37:51.928911 22695 webserver.cc:460] Webserver started at http://127.22.13.65:36173/ using document root <none> and password file <none>
I20260928 14:37:51.929149 22695 fs_manager.cc:359] Metadata directory not provided
I20260928 14:37:51.929193 22695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:37:51.929337 22695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:37:51.930462 22695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "5e8b71b5091a488f9a1275f321c90cdf"
format_stamp: "Formatted at 2026-09-28 14:37:51 on dist-test-slave-wwwb"
server_key: "30771bf905490d7f2e31da1e3ecd8a5e"
server_key_iv: "de5d2761d6de03520ae42aab257ac2b9"
server_key_version: "encryptionkey@0"
I20260928 14:37:51.930825 22695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "5e8b71b5091a488f9a1275f321c90cdf"
format_stamp: "Formatted at 2026-09-28 14:37:51 on dist-test-slave-wwwb"
server_key: "30771bf905490d7f2e31da1e3ecd8a5e"
server_key_iv: "de5d2761d6de03520ae42aab257ac2b9"
server_key_version: "encryptionkey@0"
I20260928 14:37:51.932205 22695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:37:51.932907 22710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:51.933080 22695 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:37:51.933190 22695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "5e8b71b5091a488f9a1275f321c90cdf"
format_stamp: "Formatted at 2026-09-28 14:37:51 on dist-test-slave-wwwb"
server_key: "30771bf905490d7f2e31da1e3ecd8a5e"
server_key_iv: "de5d2761d6de03520ae42aab257ac2b9"
server_key_version: "encryptionkey@0"
I20260928 14:37:51.933267 22695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:37:51.951109 22695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:37:51.953081 22695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:37:51.953258 22695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:37:51.953504 22695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:37:51.953850 22695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:37:51.953900 22695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:51.953966 22695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:37:51.953997 22695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:51.960131 22695 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:44439
I20260928 14:37:51.960177 22823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:44439 every 8 connection(s)
I20260928 14:37:51.960618 22695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:37:51.964304 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 22695
I20260928 14:37:51.964402 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:37:51.964548 22581 external_mini_cluster.cc:1428] Setting key 1a5d31d32f632755041bf03414e7a074
Sep 28 14:37:51 dist-test-slave-wwwb krb5kdc[22588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606271, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:51.971280 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:51.961440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:37161 (local address 127.22.13.126:43531)
0928 14:37:51.961490 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:51.961492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:51.961702 (+ 210us) server_negotiation.cc:407] Connection header received
0928 14:37:51.962379 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:51.962381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:51.962399 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:51.962457 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:51.963495 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:51.963996 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:51.965403 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:51.965622 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:51.965680 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:51.968338 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:51.968340 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:51.968340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:51.968349 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:51.969785 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:51.970272 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:51.970274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:51.970274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:51.970338 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:51.970604 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:51.970605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:51.970605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:51.970729 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:51.970800 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:51.971089 (+ 289us) server_negotiation.cc:299] Negotiation successful
0928 14:37:51.971187 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:37:51.971487 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:51.961420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:43531 (local address 127.22.13.65:37161)
0928 14:37:51.961575 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:51.961612 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:51.962268 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:51.962535 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:51.962548 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:51.963177 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:51.963379 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:51.963383 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:51.964098 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:51.964099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:51.965319 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:51.965323 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:51.965708 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:51.965709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:51.965872 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:51.966419 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:51.966431 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:51.968230 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:51.969891 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:51.969900 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:51.969903 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:51.970194 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:51.970404 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:51.970405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:51.970405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:51.970532 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:51.970791 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:51.970799 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:51.970955 (+ 156us) client_negotiation.cc:769] Sending connection context
0928 14:37:51.971055 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 14:37:51.971234 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260928 14:37:51.972133 22824 heartbeater.cc:347] Connected to a master server at 127.22.13.126:43531
I20260928 14:37:51.972254 22824 heartbeater.cc:464] Registering TS with master...
I20260928 14:37:51.972474 22824 heartbeater.cc:511] Master 127.22.13.126:43531 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
I20260928 14:37:51.973052 22633 ts_manager.cc:194] Registered new tserver with Master: 5e8b71b5091a488f9a1275f321c90cdf (127.22.13.65:44439)
I20260928 14:37:51.973673 22633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:37161
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:37:51.988848 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:43531
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:43339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:37:52.071019 22829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:37:52.071234 22829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:37:52.071257 22829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:37:52.072342 22829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:37:52.072379 22829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:37:52.072405 22829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:37:52.073874 22829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:43339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:43531
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:37:52.074129 22829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:37:52.074316 22829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:37:52.079263 22829 server_base.cc:1034] running on GCE node
W20260928 14:37:52.079228 22834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:52.079279 22835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:52.079216 22837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:52.079700 22829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:37:52.079949 22829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:37:52.081099 22829 hybrid_clock.cc:648] HybridClock initialized: now 1790606272081093 us; error 48 us; skew 500 ppm
Sep 28 14:37:52 dist-test-slave-wwwb krb5kdc[22588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606272, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:37:52.083824 22829 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:37:52.084283 22829 webserver.cc:460] Webserver started at http://127.22.13.66:44189/ using document root <none> and password file <none>
I20260928 14:37:52.084532 22829 fs_manager.cc:359] Metadata directory not provided
I20260928 14:37:52.084599 22829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:37:52.084753 22829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:37:52.085788 22829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "b0a35734144343c592cdbd001608c9c1"
format_stamp: "Formatted at 2026-09-28 14:37:52 on dist-test-slave-wwwb"
server_key: "37425e6b67563f2962f287abcdb7d0ae"
server_key_iv: "fe0dabbb18182f110cdc1ffb1a2e90fd"
server_key_version: "encryptionkey@0"
I20260928 14:37:52.086161 22829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "b0a35734144343c592cdbd001608c9c1"
format_stamp: "Formatted at 2026-09-28 14:37:52 on dist-test-slave-wwwb"
server_key: "37425e6b67563f2962f287abcdb7d0ae"
server_key_iv: "fe0dabbb18182f110cdc1ffb1a2e90fd"
server_key_version: "encryptionkey@0"
I20260928 14:37:52.087474 22829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:37:52.088202 22844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:52.088373 22829 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:37:52.088470 22829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "b0a35734144343c592cdbd001608c9c1"
format_stamp: "Formatted at 2026-09-28 14:37:52 on dist-test-slave-wwwb"
server_key: "37425e6b67563f2962f287abcdb7d0ae"
server_key_iv: "fe0dabbb18182f110cdc1ffb1a2e90fd"
server_key_version: "encryptionkey@0"
I20260928 14:37:52.088538 22829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:37:52.113091 22829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:37:52.115478 22829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:37:52.115660 22829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:37:52.115895 22829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:37:52.116215 22829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:37:52.116263 22829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:52.116302 22829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:37:52.116328 22829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:52.122092 22829 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:39749
I20260928 14:37:52.122164 22957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:39749 every 8 connection(s)
I20260928 14:37:52.122486 22829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:37:52.123857 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 22829
I20260928 14:37:52.123953 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:37:52.124131 22581 external_mini_cluster.cc:1428] Setting key 1d6874414d7c150348d8ad81e79dfa84
Sep 28 14:37:52 dist-test-slave-wwwb krb5kdc[22588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606272, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
I20260928 14:37:52.132484 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:52.123260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:48823 (local address 127.22.13.126:43531)
0928 14:37:52.123349 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:52.123352 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:52.123529 (+ 177us) server_negotiation.cc:407] Connection header received
0928 14:37:52.124204 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:52.124206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:52.124219 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:52.124295 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:52.125617 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:52.126220 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:52.127234 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:52.127433 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:52.127514 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:52.129659 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:52.129661 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:52.129661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:52.129673 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:52.131074 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:52.131480 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:52.131481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:52.131481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:52.131540 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:52.131813 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:52.131814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:52.131814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:52.131951 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:52.132026 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:52.132297 (+ 271us) server_negotiation.cc:299] Negotiation successful
0928 14:37:52.132390 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 14:37:52.132694 22960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:52.123229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:43531 (local address 127.22.13.66:48823)
0928 14:37:52.123404 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:52.123441 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:52.124085 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:52.124388 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:52.124392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:52.125215 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:52.125510 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:52.125516 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:52.126316 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:52.126316 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:52.127129 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:52.127131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:52.127523 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:52.127524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:52.127671 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:52.128223 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:52.128232 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:52.129561 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:52.131157 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:52.131165 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:52.131168 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:52.131405 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:52.131630 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:52.131631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:52.131632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:52.131751 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:52.132018 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:52.132025 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:52.132171 (+ 146us) client_negotiation.cc:769] Sending connection context
0928 14:37:52.132278 (+ 107us) client_negotiation.cc:240] Negotiation successful
0928 14:37:52.132442 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
I20260928 14:37:52.133208 22958 heartbeater.cc:347] Connected to a master server at 127.22.13.126:43531
I20260928 14:37:52.133308 22958 heartbeater.cc:464] Registering TS with master...
I20260928 14:37:52.133498 22958 heartbeater.cc:511] Master 127.22.13.126:43531 requested a full tablet report, sending...
I20260928 14:37:52.133882 22633 ts_manager.cc:194] Registered new tserver with Master: b0a35734144343c592cdbd001608c9c1 (127.22.13.66:39749)
I20260928 14:37:52.134287 22633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:48823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:37:52.148923 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:43531
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:43339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:37:52.233538 22963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:37:52.233810 22963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:37:52.233855 22963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:37:52.235193 22963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:37:52.235285 22963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:37:52.235335 22963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:37:52.237214 22963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:43339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:43531
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:37:52.237525 22963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:37:52.237762 22963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:37:52.243590 22971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:52.243695 22969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:52.243873 22963 server_base.cc:1034] running on GCE node
W20260928 14:37:52.243690 22968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:52.244110 22963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:37:52.244421 22963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:37:52.245589 22963 hybrid_clock.cc:648] HybridClock initialized: now 1790606272245571 us; error 29 us; skew 500 ppm
Sep 28 14:37:52 dist-test-slave-wwwb krb5kdc[22588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606272, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:37:52.248682 22963 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:37:52.249190 22963 webserver.cc:460] Webserver started at http://127.22.13.67:35157/ using document root <none> and password file <none>
I20260928 14:37:52.249424 22963 fs_manager.cc:359] Metadata directory not provided
I20260928 14:37:52.249488 22963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:37:52.249644 22963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:37:52.250886 22963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "8ee36dae8c3c4cb9893cc1e4844a90b7"
format_stamp: "Formatted at 2026-09-28 14:37:52 on dist-test-slave-wwwb"
server_key: "31eae0f4f792626e70156c7fd5b9bc64"
server_key_iv: "afbf854a70e2e4746afa6d95feb7128a"
server_key_version: "encryptionkey@0"
I20260928 14:37:52.251289 22963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "8ee36dae8c3c4cb9893cc1e4844a90b7"
format_stamp: "Formatted at 2026-09-28 14:37:52 on dist-test-slave-wwwb"
server_key: "31eae0f4f792626e70156c7fd5b9bc64"
server_key_iv: "afbf854a70e2e4746afa6d95feb7128a"
server_key_version: "encryptionkey@0"
I20260928 14:37:52.252893 22963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:37:52.253679 22978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:52.253840 22963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:37:52.253970 22963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "8ee36dae8c3c4cb9893cc1e4844a90b7"
format_stamp: "Formatted at 2026-09-28 14:37:52 on dist-test-slave-wwwb"
server_key: "31eae0f4f792626e70156c7fd5b9bc64"
server_key_iv: "afbf854a70e2e4746afa6d95feb7128a"
server_key_version: "encryptionkey@0"
I20260928 14:37:52.254057 22963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:37:52.263036 22963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:37:52.265499 22963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:37:52.265710 22963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:37:52.266155 22963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:37:52.266567 22963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:37:52.266621 22963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:52.266662 22963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:37:52.266688 22963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:52.274760 22963 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:35555
I20260928 14:37:52.274798 23091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:35555 every 8 connection(s)
I20260928 14:37:52.275144 22963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
Sep 28 14:37:52 dist-test-slave-wwwb krb5kdc[22588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606272, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:52.283661 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 22963
I20260928 14:37:52.283763 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:37:52.283932 22581 external_mini_cluster.cc:1428] Setting key 1bc0cadeddb848445a3f4655ff93964e
I20260928 14:37:52.284763 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:52.275912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:36333 (local address 127.22.13.126:43531)
0928 14:37:52.275981 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:52.275983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:52.276273 (+ 290us) server_negotiation.cc:407] Connection header received
0928 14:37:52.276759 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:52.276761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:52.276772 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:52.276825 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:52.277756 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:52.278268 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:52.279291 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:52.279486 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:52.279543 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:52.281788 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:52.281790 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:52.281790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:52.281799 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:52.283090 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:52.283542 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:52.283543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:52.283543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:52.283621 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:52.283902 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:52.283903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:52.283903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:52.284062 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:52.284134 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:52.284545 (+ 411us) server_negotiation.cc:299] Negotiation successful
0928 14:37:52.284656 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:37:52.284909 23094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:52.275913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:43531 (local address 127.22.13.67:36333)
0928 14:37:52.276147 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:52.276185 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:52.276675 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:52.276880 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:52.276884 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:52.277462 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:52.277680 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:52.277685 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:52.278353 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:52.278354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:52.279207 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:52.279210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:52.279561 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:52.279562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:52.279662 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:52.280231 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:52.280239 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:52.281677 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:52.283188 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:52.283195 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:52.283198 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:52.283446 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:52.283688 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:52.283689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:52.283689 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:52.283842 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:52.284137 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:52.284144 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:52.284328 (+ 184us) client_negotiation.cc:769] Sending connection context
0928 14:37:52.284450 (+ 122us) client_negotiation.cc:240] Negotiation successful
0928 14:37:52.284674 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":155,"threads_started":1}
I20260928 14:37:52.285483 23092 heartbeater.cc:347] Connected to a master server at 127.22.13.126:43531
I20260928 14:37:52.285571 23092 heartbeater.cc:464] Registering TS with master...
I20260928 14:37:52.285720 23092 heartbeater.cc:511] Master 127.22.13.126:43531 requested a full tablet report, sending...
I20260928 14:37:52.286100 22633 ts_manager.cc:194] Registered new tserver with Master: 8ee36dae8c3c4cb9893cc1e4844a90b7 (127.22.13.67:35555)
I20260928 14:37:52.286479 22633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:36333
I20260928 14:37:52.296072 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 14:37:52 dist-test-slave-wwwb krb5kdc[22588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606272, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 14:37:52 dist-test-slave-wwwb krb5kdc[22588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606272, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.13.65@KRBTEST.COM
I20260928 14:37:52.319732 23099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:52.312068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36154 (local address 127.22.13.65:44439)
0928 14:37:52.312264 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:52.312269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:52.312281 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:37:52.312308 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:52.312310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:52.312325 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:52.312416 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:52.312961 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:52.313693 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:52.314483 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:52.314830 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:52.314894 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:52.317224 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:52.317233 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:52.317233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:52.317240 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:52.318739 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:52.319063 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:52.319066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:52.319066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:52.319134 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:52.319290 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:52.319290 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:52.319290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:52.319431 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:52.319548 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:52.319588 (+ 40us) server_negotiation.cc:299] Negotiation successful
0928 14:37:52.319656 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":103,"threads_started":1}
W20260928 14:37:52.320118 22738 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36154
Sep 28 14:37:52 dist-test-slave-wwwb krb5kdc[22588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 14:37:52 dist-test-slave-wwwb krb5kdc[22588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.65@KRBTEST.COM
I20260928 14:37:52.342157 23099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:52.335220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36158 (local address 127.22.13.65:44439)
0928 14:37:52.335279 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:52.335282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:52.335348 (+ 66us) server_negotiation.cc:407] Connection header received
0928 14:37:52.335414 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:52.335415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:52.335427 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:52.335487 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:52.336048 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:52.336624 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:52.337363 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:52.337581 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:52.337638 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:52.339794 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:52.339796 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:52.339796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:52.339803 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:52.341196 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:52.341500 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:52.341501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:52.341502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:52.341551 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:52.341699 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:52.341699 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:52.341699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:52.341788 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:52.341868 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:52.342020 (+ 152us) server_negotiation.cc:299] Negotiation successful
0928 14:37:52.342089 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260928 14:37:52.342686 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 22695
I20260928 14:37:52.347359 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 22829
I20260928 14:37:52.352964 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 22963
I20260928 14:37:52.357470 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 22601
2026-09-28T14:37:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1900 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:37:52 dist-test-slave-wwwb krb5kdc[23106](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:37:52 dist-test-slave-wwwb krb5kdc[23106](info): set up 2 sockets
Sep 28 14:37:52 dist-test-slave-wwwb krb5kdc[23106](info): commencing operation
krb5kdc: starting...
W20260928 14:37:53.378124 22581 mini_kdc.cc:121] Time spent starting KDC: real 0.999s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:37:53 dist-test-slave-wwwb krb5kdc[23106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:37:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:37:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:37:53.496860 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:36153
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:44861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:36153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:37:53.579674 23119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:37:53.579927 23119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:37:53.579967 23119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:37:53.581209 23119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:37:53.581276 23119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:37:53.581301 23119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:37:53.581333 23119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:37:53.581362 23119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:37:53.582966 23119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:36153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:36153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:37:53.583261 23119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:37:53.583491 23119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:37:53.588542 23125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:53.588538 23127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:53.588536 23124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:53.588753 23119 server_base.cc:1034] running on GCE node
I20260928 14:37:53.589001 23119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:37:53.589344 23119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:37:53.590508 23119 hybrid_clock.cc:648] HybridClock initialized: now 1790606273590495 us; error 39 us; skew 500 ppm
Sep 28 14:37:53 dist-test-slave-wwwb krb5kdc[23106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606273, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:37:53.593072 23119 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:37:53.593567 23119 webserver.cc:460] Webserver started at http://127.22.13.126:45997/ using document root <none> and password file <none>
I20260928 14:37:53.593789 23119 fs_manager.cc:359] Metadata directory not provided
I20260928 14:37:53.593853 23119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:37:53.594055 23119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:37:53.595255 23119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "3196eb3c378e4f90b8b33c94a83e82e1"
format_stamp: "Formatted at 2026-09-28 14:37:53 on dist-test-slave-wwwb"
server_key: "379bf4b6b94d58cc4ddc28f8eaacd199"
server_key_iv: "2a943bd33b8ba151bb0243b87f77d00d"
server_key_version: "encryptionkey@0"
I20260928 14:37:53.595643 23119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "3196eb3c378e4f90b8b33c94a83e82e1"
format_stamp: "Formatted at 2026-09-28 14:37:53 on dist-test-slave-wwwb"
server_key: "379bf4b6b94d58cc4ddc28f8eaacd199"
server_key_iv: "2a943bd33b8ba151bb0243b87f77d00d"
server_key_version: "encryptionkey@0"
I20260928 14:37:53.597010 23119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:37:53.597738 23134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:53.597908 23119 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:37:53.598045 23119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "3196eb3c378e4f90b8b33c94a83e82e1"
format_stamp: "Formatted at 2026-09-28 14:37:53 on dist-test-slave-wwwb"
server_key: "379bf4b6b94d58cc4ddc28f8eaacd199"
server_key_iv: "2a943bd33b8ba151bb0243b87f77d00d"
server_key_version: "encryptionkey@0"
I20260928 14:37:53.598129 23119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:37:53.609151 23119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:37:53.611137 23119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:37:53.611305 23119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:37:53.615487 23119 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:36153
I20260928 14:37:53.615528 23186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:36153 every 8 connection(s)
I20260928 14:37:53.615921 23119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:37:53.616802 23187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:53.619161 23187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1: Bootstrap starting.
I20260928 14:37:53.619767 23187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:53.620065 23187 log.cc:824] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1: Log is configured to *not* fsync() on all Append() calls
I20260928 14:37:53.620797 23187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1: No bootstrap required, opened a new log
I20260928 14:37:53.622313 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 23119
I20260928 14:37:53.622412 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:37:53.622584 22581 external_mini_cluster.cc:1428] Setting key 1db1de9c936772e667f602d2c086fbb3
I20260928 14:37:53.622565 23187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3196eb3c378e4f90b8b33c94a83e82e1" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36153 } }
I20260928 14:37:53.622742 23187 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:53.622804 23187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3196eb3c378e4f90b8b33c94a83e82e1, State: Initialized, Role: FOLLOWER
I20260928 14:37:53.622958 23187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3196eb3c378e4f90b8b33c94a83e82e1" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36153 } }
I20260928 14:37:53.623055 23187 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:37:53.623098 23187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:37:53.623144 23187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:53.624331 23187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3196eb3c378e4f90b8b33c94a83e82e1" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36153 } }
I20260928 14:37:53.624471 23187 leader_election.cc:302] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3196eb3c378e4f90b8b33c94a83e82e1; no voters:
I20260928 14:37:53.624665 23187 leader_election.cc:288] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:37:53.624693 23192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:37:53.624799 23192 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [term 1 LEADER]: Becoming Leader. State: Replica: 3196eb3c378e4f90b8b33c94a83e82e1, State: Running, Role: LEADER
I20260928 14:37:53.624879 23192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3196eb3c378e4f90b8b33c94a83e82e1" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36153 } }
I20260928 14:37:53.624927 23187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:37:53.625203 23192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3196eb3c378e4f90b8b33c94a83e82e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3196eb3c378e4f90b8b33c94a83e82e1" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36153 } } }
I20260928 14:37:53.625277 23192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [sys.catalog]: This master's current role is: LEADER
I20260928 14:37:53.625393 23192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3196eb3c378e4f90b8b33c94a83e82e1. Latest consensus state: current_term: 1 leader_uuid: "3196eb3c378e4f90b8b33c94a83e82e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3196eb3c378e4f90b8b33c94a83e82e1" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36153 } } }
I20260928 14:37:53.625460 23192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1 [sys.catalog]: This master's current role is: LEADER
I20260928 14:37:53.625590 23197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:37:53.626322 23197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:37:53.627969 23197 catalog_manager.cc:1347] Generated new cluster ID: 1ee49791a015487a8268766b47248d90
I20260928 14:37:53.628021 23197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:37:53 dist-test-slave-wwwb krb5kdc[23106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:53.634231 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:53.623091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56598 (local address 127.22.13.126:36153)
0928 14:37:53.623281 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:53.623290 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:53.623355 (+ 65us) server_negotiation.cc:407] Connection header received
0928 14:37:53.623456 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:53.623462 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:53.623511 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:53.623620 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:53.624467 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:53.625822 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:53.626826 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:53.627405 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:53.627511 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:53.630502 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:53.630506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:53.630506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:53.630519 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:53.632492 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:53.632938 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:53.632941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:53.632941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:53.633061 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:53.633280 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:53.633281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:53.633281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:53.633578 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:53.633736 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:53.633855 (+ 119us) server_negotiation.cc:299] Negotiation successful
0928 14:37:53.634028 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":103,"threads_started":1}
I20260928 14:37:53.639570 23197 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:37:53.640089 23197 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:37:53.642721 23197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3196eb3c378e4f90b8b33c94a83e82e1: Generated new TSK 0
I20260928 14:37:53.642899 23197 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:37:53.672019 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36153
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:44861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:37:53.754372 23213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:37:53.754591 23213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:37:53.754616 23213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:37:53.755759 23213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:37:53.755795 23213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:37:53.755822 23213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:37:53.757417 23213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36153
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:37:53.757622 23213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:37:53.757812 23213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:37:53.762841 23218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:53.762841 23221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:53.762842 23219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:53.763185 23213 server_base.cc:1034] running on GCE node
I20260928 14:37:53.763408 23213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:37:53.763664 23213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:37:53.764832 23213 hybrid_clock.cc:648] HybridClock initialized: now 1790606273764815 us; error 35 us; skew 500 ppm
Sep 28 14:37:53 dist-test-slave-wwwb krb5kdc[23106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606273, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:37:53.767506 23213 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:37:53.768000 23213 webserver.cc:460] Webserver started at http://127.22.13.65:33091/ using document root <none> and password file <none>
I20260928 14:37:53.768237 23213 fs_manager.cc:359] Metadata directory not provided
I20260928 14:37:53.768302 23213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:37:53.768452 23213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:37:53.769526 23213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "c248054bcb484816a88ef90001d46b13"
format_stamp: "Formatted at 2026-09-28 14:37:53 on dist-test-slave-wwwb"
server_key: "46857d334c259e731c9739c1bd22552b"
server_key_iv: "93fe68cefea480b5d07326f75ebc6ff2"
server_key_version: "encryptionkey@0"
I20260928 14:37:53.769886 23213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "c248054bcb484816a88ef90001d46b13"
format_stamp: "Formatted at 2026-09-28 14:37:53 on dist-test-slave-wwwb"
server_key: "46857d334c259e731c9739c1bd22552b"
server_key_iv: "93fe68cefea480b5d07326f75ebc6ff2"
server_key_version: "encryptionkey@0"
I20260928 14:37:53.771240 23213 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:37:53.771935 23228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:53.772104 23213 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:37:53.772202 23213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "c248054bcb484816a88ef90001d46b13"
format_stamp: "Formatted at 2026-09-28 14:37:53 on dist-test-slave-wwwb"
server_key: "46857d334c259e731c9739c1bd22552b"
server_key_iv: "93fe68cefea480b5d07326f75ebc6ff2"
server_key_version: "encryptionkey@0"
I20260928 14:37:53.772279 23213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:37:53.799757 23213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:37:53.801800 23213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:37:53.802003 23213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:37:53.802230 23213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:37:53.802488 23213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:37:53.802520 23213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:53.802546 23213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:37:53.802557 23213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:53.808367 23213 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:36845
I20260928 14:37:53.808413 23341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:36845 every 8 connection(s)
I20260928 14:37:53.808751 23213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
Sep 28 14:37:53 dist-test-slave-wwwb krb5kdc[23106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606273, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:53.817468 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 23213
I20260928 14:37:53.817579 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:37:53.817775 22581 external_mini_cluster.cc:1428] Setting key 6caf5719660fb45936bd13eb97087f01
I20260928 14:37:53.818892 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:53.809456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:44921 (local address 127.22.13.126:36153)
0928 14:37:53.809542 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:53.809544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:53.809792 (+ 248us) server_negotiation.cc:407] Connection header received
0928 14:37:53.810849 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:53.810850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:53.810869 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:53.810922 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:53.811631 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:53.812144 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:53.813204 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:53.813407 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:53.813495 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:53.815909 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:53.815911 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:53.815911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:53.815919 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:53.817159 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:53.817717 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:53.817718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:53.817718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:53.817773 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:53.818099 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:53.818100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:53.818101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:53.818244 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:53.818309 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:53.818656 (+ 347us) server_negotiation.cc:299] Negotiation successful
0928 14:37:53.818758 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 14:37:53.818907 23344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:53.809449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36153 (local address 127.22.13.65:44921)
0928 14:37:53.809632 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:53.809693 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:53.810724 (+ 1031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:53.811001 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:53.811005 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:53.811304 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:53.811521 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:53.811526 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:53.812257 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:53.812258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:53.813109 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:53.813112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:53.813515 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:53.813516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:53.813650 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:53.814252 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:53.814261 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:53.815806 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:53.817234 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:53.817240 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:53.817243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:53.817641 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:53.817844 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:53.817845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:53.817845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:53.818021 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:53.818320 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:53.818326 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:53.818509 (+ 183us) client_negotiation.cc:769] Sending connection context
0928 14:37:53.818594 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 14:37:53.818732 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":71,"threads_started":1}
I20260928 14:37:53.819443 23342 heartbeater.cc:347] Connected to a master server at 127.22.13.126:36153
I20260928 14:37:53.819566 23342 heartbeater.cc:464] Registering TS with master...
I20260928 14:37:53.819754 23342 heartbeater.cc:511] Master 127.22.13.126:36153 requested a full tablet report, sending...
I20260928 14:37:53.820309 23151 ts_manager.cc:194] Registered new tserver with Master: c248054bcb484816a88ef90001d46b13 (127.22.13.65:36845)
I20260928 14:37:53.820987 23151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:44921
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:37:53.845086 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36153
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:44861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:37:53.925945 23347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:37:53.926199 23347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:37:53.926240 23347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:37:53.927500 23347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:37:53.927570 23347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:37:53.927619 23347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:37:53.929210 23347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36153
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:37:53.929486 23347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:37:53.929723 23347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:37:53.934628 23355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:53.934729 23353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:53.934628 23352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:53.934864 23347 server_base.cc:1034] running on GCE node
I20260928 14:37:53.935074 23347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:37:53.935364 23347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:37:53.936527 23347 hybrid_clock.cc:648] HybridClock initialized: now 1790606273936515 us; error 27 us; skew 500 ppm
Sep 28 14:37:53 dist-test-slave-wwwb krb5kdc[23106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606273, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:37:53.939055 23347 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:37:53.939555 23347 webserver.cc:460] Webserver started at http://127.22.13.66:34329/ using document root <none> and password file <none>
I20260928 14:37:53.939741 23347 fs_manager.cc:359] Metadata directory not provided
I20260928 14:37:53.939780 23347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:37:53.939870 23347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:37:53.940884 23347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "9db4549051ff4d3a979c7fd13b7dfaa3"
format_stamp: "Formatted at 2026-09-28 14:37:53 on dist-test-slave-wwwb"
server_key: "7141bc87259262f4481a5d5124ea066b"
server_key_iv: "50830e3b44d5e742854a0b423490464a"
server_key_version: "encryptionkey@0"
I20260928 14:37:53.941201 23347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "9db4549051ff4d3a979c7fd13b7dfaa3"
format_stamp: "Formatted at 2026-09-28 14:37:53 on dist-test-slave-wwwb"
server_key: "7141bc87259262f4481a5d5124ea066b"
server_key_iv: "50830e3b44d5e742854a0b423490464a"
server_key_version: "encryptionkey@0"
I20260928 14:37:53.942612 23347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260928 14:37:53.943297 23362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:53.943464 23347 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:37:53.943553 23347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "9db4549051ff4d3a979c7fd13b7dfaa3"
format_stamp: "Formatted at 2026-09-28 14:37:53 on dist-test-slave-wwwb"
server_key: "7141bc87259262f4481a5d5124ea066b"
server_key_iv: "50830e3b44d5e742854a0b423490464a"
server_key_version: "encryptionkey@0"
I20260928 14:37:53.943614 23347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:37:53.961396 23347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:37:53.963348 23347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:37:53.963464 23347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:37:53.963655 23347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:37:53.963981 23347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:37:53.964022 23347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:53.964069 23347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:37:53.964108 23347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:53.969730 23347 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:40387
I20260928 14:37:53.969818 23475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:40387 every 8 connection(s)
I20260928 14:37:53.970225 23347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
Sep 28 14:37:53 dist-test-slave-wwwb krb5kdc[23106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606273, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:53.979635 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:53.971012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:34029 (local address 127.22.13.126:36153)
0928 14:37:53.971082 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:53.971084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:53.971295 (+ 211us) server_negotiation.cc:407] Connection header received
0928 14:37:53.972270 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:53.972272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:53.972285 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:53.972344 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:53.973017 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:53.973527 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:53.974608 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:53.974817 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:53.974899 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:53.977165 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:53.977166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:53.977167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:53.977175 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:53.978459 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:53.978841 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:53.978841 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:53.978841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:53.978891 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:53.979106 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:53.979106 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:53.979107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:53.979213 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:53.979263 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:53.979497 (+ 234us) server_negotiation.cc:299] Negotiation successful
0928 14:37:53.979566 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 14:37:53.979861 23478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:53.971014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36153 (local address 127.22.13.66:34029)
0928 14:37:53.971169 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:53.971206 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:53.972136 (+ 930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:53.972422 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:53.972426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:53.972695 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:53.972900 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:53.972905 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:53.973623 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:53.973625 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:53.974515 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:53.974518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:53.974910 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:53.974911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:53.975036 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:53.975605 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:53.975615 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:53.977065 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:53.978539 (+ 1474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:53.978545 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:53.978547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:53.978776 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:53.978946 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:53.978946 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:53.978947 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:53.979053 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:53.979263 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:53.979267 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:53.979393 (+ 126us) client_negotiation.cc:769] Sending connection context
0928 14:37:53.979476 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 14:37:53.979612 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260928 14:37:53.980026 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 23347
I20260928 14:37:53.980111 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:37:53.980281 22581 external_mini_cluster.cc:1428] Setting key 5b6b96ad0fb848de6230777b0ec02c41
I20260928 14:37:53.980335 23476 heartbeater.cc:347] Connected to a master server at 127.22.13.126:36153
I20260928 14:37:53.980458 23476 heartbeater.cc:464] Registering TS with master...
I20260928 14:37:53.980657 23476 heartbeater.cc:511] Master 127.22.13.126:36153 requested a full tablet report, sending...
I20260928 14:37:53.981109 23151 ts_manager.cc:194] Registered new tserver with Master: 9db4549051ff4d3a979c7fd13b7dfaa3 (127.22.13.66:40387)
I20260928 14:37:53.981479 23151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:34029
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:37:54.004855 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36153
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:44861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:37:54.088181 23481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:37:54.088409 23481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:37:54.088434 23481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:37:54.089620 23481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:37:54.089661 23481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:37:54.089689 23481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:37:54.091168 23481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36153
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:37:54.091379 23481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:37:54.091569 23481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:37:54.096447 23486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:54.096447 23489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:54.096472 23487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:54.096709 23481 server_base.cc:1034] running on GCE node
I20260928 14:37:54.096927 23481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:37:54.097119 23481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:37:54.098253 23481 hybrid_clock.cc:648] HybridClock initialized: now 1790606274098240 us; error 28 us; skew 500 ppm
Sep 28 14:37:54 dist-test-slave-wwwb krb5kdc[23106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606274, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:37:54.100811 23481 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:37:54.101323 23481 webserver.cc:460] Webserver started at http://127.22.13.67:42381/ using document root <none> and password file <none>
I20260928 14:37:54.101563 23481 fs_manager.cc:359] Metadata directory not provided
I20260928 14:37:54.101626 23481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:37:54.101763 23481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:37:54.102867 23481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "4498f807e6e143c9b8e106339350933b"
format_stamp: "Formatted at 2026-09-28 14:37:54 on dist-test-slave-wwwb"
server_key: "1eed16dbafb82c6da5b627c24c47acee"
server_key_iv: "abee7271a9ed80db2201f06a33ce6a28"
server_key_version: "encryptionkey@0"
I20260928 14:37:54.103271 23481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "4498f807e6e143c9b8e106339350933b"
format_stamp: "Formatted at 2026-09-28 14:37:54 on dist-test-slave-wwwb"
server_key: "1eed16dbafb82c6da5b627c24c47acee"
server_key_iv: "abee7271a9ed80db2201f06a33ce6a28"
server_key_version: "encryptionkey@0"
I20260928 14:37:54.104628 23481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:37:54.105309 23496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:54.105468 23481 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:37:54.105564 23481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "4498f807e6e143c9b8e106339350933b"
format_stamp: "Formatted at 2026-09-28 14:37:54 on dist-test-slave-wwwb"
server_key: "1eed16dbafb82c6da5b627c24c47acee"
server_key_iv: "abee7271a9ed80db2201f06a33ce6a28"
server_key_version: "encryptionkey@0"
I20260928 14:37:54.105641 23481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:37:54.125150 23481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:37:54.127295 23481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:37:54.127460 23481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:37:54.127749 23481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:37:54.128103 23481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:37:54.128161 23481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:54.128203 23481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:37:54.128232 23481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:54.134119 23481 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:46675
I20260928 14:37:54.134163 23609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:46675 every 8 connection(s)
I20260928 14:37:54.134538 23481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:37:54.140077 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 23481
I20260928 14:37:54.140162 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:37:54.140322 22581 external_mini_cluster.cc:1428] Setting key 34c73cf1859206478f9c0de8666d86c4
Sep 28 14:37:54 dist-test-slave-wwwb krb5kdc[23106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606274, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:54.144169 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:54.135287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:46271 (local address 127.22.13.126:36153)
0928 14:37:54.135356 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:54.135358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:54.135578 (+ 220us) server_negotiation.cc:407] Connection header received
0928 14:37:54.136490 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:54.136491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:54.136500 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:54.136540 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:54.137150 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.137637 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.138665 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.138842 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.138900 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:54.141366 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:54.141367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:54.141367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:54.141374 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:54.142702 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:54.143129 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:54.143131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:54.143131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:54.143202 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:54.143476 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:54.143477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:54.143477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:54.143603 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:54.143698 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:54.143982 (+ 284us) server_negotiation.cc:299] Negotiation successful
0928 14:37:54.144076 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:37:54.144379 23612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:54.135283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36153 (local address 127.22.13.67:46271)
0928 14:37:54.135443 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:54.135486 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:54.136378 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:54.136612 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:54.136615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:54.136871 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:54.137078 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:54.137082 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.137716 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.137717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:54.138585 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:54.138588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.138910 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.138910 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:54.139012 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:54.139584 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:54.139592 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:54.141287 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:54.142798 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:54.142806 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:54.142808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:54.143050 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:54.143276 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:54.143277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:54.143277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:54.143408 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:54.143688 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:54.143695 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:54.143846 (+ 151us) client_negotiation.cc:769] Sending connection context
0928 14:37:54.143959 (+ 113us) client_negotiation.cc:240] Negotiation successful
0928 14:37:54.144127 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260928 14:37:54.144829 23610 heartbeater.cc:347] Connected to a master server at 127.22.13.126:36153
I20260928 14:37:54.144934 23610 heartbeater.cc:464] Registering TS with master...
I20260928 14:37:54.145128 23610 heartbeater.cc:511] Master 127.22.13.126:36153 requested a full tablet report, sending...
I20260928 14:37:54.145511 23151 ts_manager.cc:194] Registered new tserver with Master: 4498f807e6e143c9b8e106339350933b (127.22.13.67:46675)
I20260928 14:37:54.145877 23151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:46271
I20260928 14:37:54.152210 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:37:54.159520 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:54.153761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56600 (local address 127.22.13.126:36153)
0928 14:37:54.153822 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:54.153824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:54.153981 (+ 157us) server_negotiation.cc:407] Connection header received
0928 14:37:54.154068 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:54.154069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:54.154079 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:54.154127 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:54.154661 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.155146 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.155806 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.155984 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.156035 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:54.157133 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:54.157134 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:54.157134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:54.157140 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:54.158550 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:54.158842 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:54.158843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:54.158843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:54.158893 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:54.159050 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:54.159050 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:54.159051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:54.159215 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:54.159292 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:54.159384 (+ 92us) server_negotiation.cc:299] Negotiation successful
0928 14:37:54.159447 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 14:37:54.161682 23151 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:37:54.163280 23151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:37:54.169806 23626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:54.166024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59062 (local address 127.22.13.67:46675)
0928 14:37:54.166196 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:54.166200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:54.166209 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:37:54.166245 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:54.166247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:54.166260 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:54.166478 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:54.166554 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.167635 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.168851 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.169340 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.169451 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:54.169516 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:54.169670 (+ 154us) server_negotiation.cc:299] Negotiation successful
0928 14:37:54.169709 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260928 14:37:54.169806 23622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:54.165541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:40387 (local address 127.0.0.1:55692)
0928 14:37:54.165713 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:54.165755 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:54.165825 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:54.166249 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:54.166249 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:54.166252 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:54.166306 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:54.166308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.167338 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.167339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:54.168578 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:54.168581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.169527 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.169528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:54.169594 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:54.169599 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:37:54.169627 (+ 28us) client_negotiation.cc:240] Negotiation successful
0928 14:37:54.169694 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260928 14:37:54.169808 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:54.165541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:46675 (local address 127.0.0.1:59062)
0928 14:37:54.165711 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:54.165761 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:54.165825 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:54.166359 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:54.166359 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:54.166362 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:54.166409 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:54.166410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.167747 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.167748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:54.168773 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:54.168774 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.169453 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.169454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:54.169543 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:54.169557 (+ 14us) client_negotiation.cc:769] Sending connection context
0928 14:37:54.169652 (+ 95us) client_negotiation.cc:240] Negotiation successful
0928 14:37:54.169700 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":65,"threads_started":1}
I20260928 14:37:54.170523 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:54.165822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55692 (local address 127.22.13.66:40387)
0928 14:37:54.166109 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:54.166114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:54.166124 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:37:54.166162 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:54.166164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:54.166177 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:54.166249 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:54.166427 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.167232 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.168672 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.169430 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.170306 (+ 876us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:54.170372 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:54.170398 (+ 26us) server_negotiation.cc:299] Negotiation successful
0928 14:37:54.170440 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":83,"threads_started":1}
I20260928 14:37:54.170825 23544 tablet_service.cc:1467] Processing CreateTablet for tablet 26de00009bb945228d12a97f068b5576 (DEFAULT_TABLE table=test-table [id=a9c76ec0b68444c081690831401ed7c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:37:54.171161 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:54.165550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:36845 (local address 127.0.0.1:35622)
0928 14:37:54.165927 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:54.165936 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:54.165994 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:54.166210 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:54.166214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:54.166217 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:54.166306 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:54.166311 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.168869 (+ 2558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.168870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:54.169896 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:54.169899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.170915 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.170916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:54.170979 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:54.170985 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 14:37:54.171048 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 14:37:54.171092 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":94,"threads_started":1}
I20260928 14:37:54.171176 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:54.165822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35622 (local address 127.22.13.65:36845)
0928 14:37:54.166008 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:54.166013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:54.166023 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:37:54.166065 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:54.166067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:54.166079 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:54.166387 (+ 308us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:54.166461 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.167632 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.170029 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.170816 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.170911 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:54.170966 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:54.171062 (+ 96us) server_negotiation.cc:299] Negotiation successful
0928 14:37:54.171101 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":82,"threads_started":1}
I20260928 14:37:54.171211 23544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26de00009bb945228d12a97f068b5576. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:54.171149 23410 tablet_service.cc:1467] Processing CreateTablet for tablet 26de00009bb945228d12a97f068b5576 (DEFAULT_TABLE table=test-table [id=a9c76ec0b68444c081690831401ed7c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:37:54.171461 23410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26de00009bb945228d12a97f068b5576. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:54.171731 23276 tablet_service.cc:1467] Processing CreateTablet for tablet 26de00009bb945228d12a97f068b5576 (DEFAULT_TABLE table=test-table [id=a9c76ec0b68444c081690831401ed7c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:37:54.171963 23276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26de00009bb945228d12a97f068b5576. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:54.174683 23628 tablet_bootstrap.cc:492] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3: Bootstrap starting.
I20260928 14:37:54.174947 23627 tablet_bootstrap.cc:492] T 26de00009bb945228d12a97f068b5576 P c248054bcb484816a88ef90001d46b13: Bootstrap starting.
I20260928 14:37:54.175490 23628 tablet_bootstrap.cc:654] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:54.175642 23627 tablet_bootstrap.cc:654] T 26de00009bb945228d12a97f068b5576 P c248054bcb484816a88ef90001d46b13: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:54.176159 23628 log.cc:824] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3: Log is configured to *not* fsync() on all Append() calls
I20260928 14:37:54.176195 23627 log.cc:824] T 26de00009bb945228d12a97f068b5576 P c248054bcb484816a88ef90001d46b13: Log is configured to *not* fsync() on all Append() calls
I20260928 14:37:54.176520 23629 tablet_bootstrap.cc:492] T 26de00009bb945228d12a97f068b5576 P 4498f807e6e143c9b8e106339350933b: Bootstrap starting.
I20260928 14:37:54.176962 23627 tablet_bootstrap.cc:492] T 26de00009bb945228d12a97f068b5576 P c248054bcb484816a88ef90001d46b13: No bootstrap required, opened a new log
I20260928 14:37:54.177021 23627 ts_tablet_manager.cc:1397] T 26de00009bb945228d12a97f068b5576 P c248054bcb484816a88ef90001d46b13: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:37:54.177157 23629 tablet_bootstrap.cc:654] T 26de00009bb945228d12a97f068b5576 P 4498f807e6e143c9b8e106339350933b: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:54.177265 23628 tablet_bootstrap.cc:492] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3: No bootstrap required, opened a new log
I20260928 14:37:54.177341 23628 ts_tablet_manager.cc:1397] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:37:54.177749 23629 log.cc:824] T 26de00009bb945228d12a97f068b5576 P 4498f807e6e143c9b8e106339350933b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:37:54.178545 23629 tablet_bootstrap.cc:492] T 26de00009bb945228d12a97f068b5576 P 4498f807e6e143c9b8e106339350933b: No bootstrap required, opened a new log
I20260928 14:37:54.178615 23629 ts_tablet_manager.cc:1397] T 26de00009bb945228d12a97f068b5576 P 4498f807e6e143c9b8e106339350933b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260928 14:37:54.178951 23627 raft_consensus.cc:348] T 26de00009bb945228d12a97f068b5576 P c248054bcb484816a88ef90001d46b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498f807e6e143c9b8e106339350933b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46675 } } peers { permanent_uuid: "9db4549051ff4d3a979c7fd13b7dfaa3" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 40387 } } peers { permanent_uuid: "c248054bcb484816a88ef90001d46b13" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 36845 } }
I20260928 14:37:54.179112 23627 raft_consensus.cc:374] T 26de00009bb945228d12a97f068b5576 P c248054bcb484816a88ef90001d46b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:54.179154 23627 raft_consensus.cc:729] T 26de00009bb945228d12a97f068b5576 P c248054bcb484816a88ef90001d46b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c248054bcb484816a88ef90001d46b13, State: Initialized, Role: FOLLOWER
I20260928 14:37:54.179281 23627 consensus_queue.cc:260] T 26de00009bb945228d12a97f068b5576 P c248054bcb484816a88ef90001d46b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498f807e6e143c9b8e106339350933b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46675 } } peers { permanent_uuid: "9db4549051ff4d3a979c7fd13b7dfaa3" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 40387 } } peers { permanent_uuid: "c248054bcb484816a88ef90001d46b13" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 36845 } }
I20260928 14:37:54.179432 23628 raft_consensus.cc:348] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498f807e6e143c9b8e106339350933b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46675 } } peers { permanent_uuid: "9db4549051ff4d3a979c7fd13b7dfaa3" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 40387 } } peers { permanent_uuid: "c248054bcb484816a88ef90001d46b13" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 36845 } }
I20260928 14:37:54.179584 23627 ts_tablet_manager.cc:1428] T 26de00009bb945228d12a97f068b5576 P c248054bcb484816a88ef90001d46b13: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:37:54.179611 23628 raft_consensus.cc:374] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:54.179647 23628 raft_consensus.cc:729] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db4549051ff4d3a979c7fd13b7dfaa3, State: Initialized, Role: FOLLOWER
I20260928 14:37:54.179585 23342 heartbeater.cc:503] Master 127.22.13.126:36153 was elected leader, sending a full tablet report...
I20260928 14:37:54.179756 23628 consensus_queue.cc:260] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498f807e6e143c9b8e106339350933b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46675 } } peers { permanent_uuid: "9db4549051ff4d3a979c7fd13b7dfaa3" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 40387 } } peers { permanent_uuid: "c248054bcb484816a88ef90001d46b13" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 36845 } }
I20260928 14:37:54.180030 23628 ts_tablet_manager.cc:1428] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:37:54.180055 23476 heartbeater.cc:503] Master 127.22.13.126:36153 was elected leader, sending a full tablet report...
I20260928 14:37:54.181221 23629 raft_consensus.cc:348] T 26de00009bb945228d12a97f068b5576 P 4498f807e6e143c9b8e106339350933b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498f807e6e143c9b8e106339350933b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46675 } } peers { permanent_uuid: "9db4549051ff4d3a979c7fd13b7dfaa3" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 40387 } } peers { permanent_uuid: "c248054bcb484816a88ef90001d46b13" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 36845 } }
I20260928 14:37:54.181370 23629 raft_consensus.cc:374] T 26de00009bb945228d12a97f068b5576 P 4498f807e6e143c9b8e106339350933b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:54.181392 23629 raft_consensus.cc:729] T 26de00009bb945228d12a97f068b5576 P 4498f807e6e143c9b8e106339350933b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4498f807e6e143c9b8e106339350933b, State: Initialized, Role: FOLLOWER
I20260928 14:37:54.181489 23629 consensus_queue.cc:260] T 26de00009bb945228d12a97f068b5576 P 4498f807e6e143c9b8e106339350933b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498f807e6e143c9b8e106339350933b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46675 } } peers { permanent_uuid: "9db4549051ff4d3a979c7fd13b7dfaa3" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 40387 } } peers { permanent_uuid: "c248054bcb484816a88ef90001d46b13" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 36845 } }
I20260928 14:37:54.181730 23629 ts_tablet_manager.cc:1428] T 26de00009bb945228d12a97f068b5576 P 4498f807e6e143c9b8e106339350933b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 14:37:54.181754 23610 heartbeater.cc:503] Master 127.22.13.126:36153 was elected leader, sending a full tablet report...
I20260928 14:37:54.187026 23634 raft_consensus.cc:482] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:37:54.187145 23634 raft_consensus.cc:504] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498f807e6e143c9b8e106339350933b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46675 } } peers { permanent_uuid: "9db4549051ff4d3a979c7fd13b7dfaa3" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 40387 } } peers { permanent_uuid: "c248054bcb484816a88ef90001d46b13" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 36845 } }
I20260928 14:37:54.187422 23634 leader_election.cc:288] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4498f807e6e143c9b8e106339350933b (127.22.13.67:46675), c248054bcb484816a88ef90001d46b13 (127.22.13.65:36845)
I20260928 14:37:54.191247 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:54.187705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:33265 (local address 127.22.13.65:36845)
0928 14:37:54.187751 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:54.187753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:54.187762 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:37:54.187792 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:54.187793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:54.187801 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:54.187876 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:54.188041 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.188826 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.190350 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.190885 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.190953 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:54.190961 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:54.191128 (+ 167us) server_negotiation.cc:299] Negotiation successful
0928 14:37:54.191167 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:37:54.191253 23478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:54.187581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:36845 (local address 127.22.13.66:33265)
0928 14:37:54.187644 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:54.187659 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:54.187713 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:54.187877 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:54.187877 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:54.187879 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:54.187968 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:54.187970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.188914 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.188915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:54.190253 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:54.190258 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.190969 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.190970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:54.191059 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:54.191062 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:37:54.191115 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 14:37:54.191162 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":57}
I20260928 14:37:54.191336 23626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:54.187797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:47015 (local address 127.22.13.67:46675)
0928 14:37:54.187858 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:54.187860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:54.187868 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:37:54.187891 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:54.187891 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:54.187902 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:54.187954 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:54.188089 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.188826 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.190448 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.190975 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.191057 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:54.191065 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:54.191228 (+ 163us) server_negotiation.cc:299] Negotiation successful
0928 14:37:54.191268 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:37:54.191545 23636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:54.187581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:46675 (local address 127.22.13.66:47015)
0928 14:37:54.187757 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:54.187767 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:54.187810 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:54.187966 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:54.187966 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:54.187967 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:54.188027 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:54.188028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.188910 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.188911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:54.190253 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:54.190258 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.191060 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.191061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:54.191154 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:54.191159 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:37:54.191215 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 14:37:54.191259 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":40,"thread_start_us":96,"threads_started":1}
I20260928 14:37:54.191637 23296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26de00009bb945228d12a97f068b5576" candidate_uuid: "9db4549051ff4d3a979c7fd13b7dfaa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c248054bcb484816a88ef90001d46b13" is_pre_election: true
I20260928 14:37:54.191807 23296 raft_consensus.cc:2454] T 26de00009bb945228d12a97f068b5576 P c248054bcb484816a88ef90001d46b13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9db4549051ff4d3a979c7fd13b7dfaa3 in term 0.
I20260928 14:37:54.191855 23564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26de00009bb945228d12a97f068b5576" candidate_uuid: "9db4549051ff4d3a979c7fd13b7dfaa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4498f807e6e143c9b8e106339350933b" is_pre_election: true
I20260928 14:37:54.192010 23564 raft_consensus.cc:2454] T 26de00009bb945228d12a97f068b5576 P 4498f807e6e143c9b8e106339350933b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9db4549051ff4d3a979c7fd13b7dfaa3 in term 0.
I20260928 14:37:54.192022 23365 leader_election.cc:302] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9db4549051ff4d3a979c7fd13b7dfaa3, c248054bcb484816a88ef90001d46b13; no voters:
I20260928 14:37:54.192148 23634 raft_consensus.cc:2781] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:37:54.192203 23634 raft_consensus.cc:482] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:37:54.192237 23634 raft_consensus.cc:3037] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:54.192907 23634 raft_consensus.cc:504] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498f807e6e143c9b8e106339350933b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46675 } } peers { permanent_uuid: "9db4549051ff4d3a979c7fd13b7dfaa3" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 40387 } } peers { permanent_uuid: "c248054bcb484816a88ef90001d46b13" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 36845 } }
I20260928 14:37:54.193058 23634 leader_election.cc:288] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [CANDIDATE]: Term 1 election: Requested vote from peers 4498f807e6e143c9b8e106339350933b (127.22.13.67:46675), c248054bcb484816a88ef90001d46b13 (127.22.13.65:36845)
I20260928 14:37:54.193199 23564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26de00009bb945228d12a97f068b5576" candidate_uuid: "9db4549051ff4d3a979c7fd13b7dfaa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4498f807e6e143c9b8e106339350933b"
I20260928 14:37:54.193233 23296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26de00009bb945228d12a97f068b5576" candidate_uuid: "9db4549051ff4d3a979c7fd13b7dfaa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c248054bcb484816a88ef90001d46b13"
I20260928 14:37:54.193290 23564 raft_consensus.cc:3037] T 26de00009bb945228d12a97f068b5576 P 4498f807e6e143c9b8e106339350933b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:54.193297 23296 raft_consensus.cc:3037] T 26de00009bb945228d12a97f068b5576 P c248054bcb484816a88ef90001d46b13 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:54.194067 23296 raft_consensus.cc:2454] T 26de00009bb945228d12a97f068b5576 P c248054bcb484816a88ef90001d46b13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9db4549051ff4d3a979c7fd13b7dfaa3 in term 1.
I20260928 14:37:54.194082 23564 raft_consensus.cc:2454] T 26de00009bb945228d12a97f068b5576 P 4498f807e6e143c9b8e106339350933b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9db4549051ff4d3a979c7fd13b7dfaa3 in term 1.
I20260928 14:37:54.194216 23365 leader_election.cc:302] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9db4549051ff4d3a979c7fd13b7dfaa3, c248054bcb484816a88ef90001d46b13; no voters:
I20260928 14:37:54.194300 23634 raft_consensus.cc:2781] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:37:54.194473 23634 raft_consensus.cc:686] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [term 1 LEADER]: Becoming Leader. State: Replica: 9db4549051ff4d3a979c7fd13b7dfaa3, State: Running, Role: LEADER
I20260928 14:37:54.194578 23634 consensus_queue.cc:237] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498f807e6e143c9b8e106339350933b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46675 } } peers { permanent_uuid: "9db4549051ff4d3a979c7fd13b7dfaa3" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 40387 } } peers { permanent_uuid: "c248054bcb484816a88ef90001d46b13" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 36845 } }
I20260928 14:37:54.195421 23151 catalog_manager.cc:5495] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9db4549051ff4d3a979c7fd13b7dfaa3 (127.22.13.66). New cstate: current_term: 1 leader_uuid: "9db4549051ff4d3a979c7fd13b7dfaa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4498f807e6e143c9b8e106339350933b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9db4549051ff4d3a979c7fd13b7dfaa3" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 40387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c248054bcb484816a88ef90001d46b13" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 36845 } health_report { overall_health: UNKNOWN } } }
Sep 28 14:37:54 dist-test-slave-wwwb krb5kdc[23106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.65@KRBTEST.COM
I20260928 14:37:54.205170 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:54.197698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35624 (local address 127.22.13.65:36845)
0928 14:37:54.197738 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:54.197740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:54.197835 (+ 95us) server_negotiation.cc:407] Connection header received
0928 14:37:54.197884 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:54.197884 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:54.197893 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:54.197996 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:54.198536 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.198999 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.199693 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.199873 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.199924 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:54.202366 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:54.202380 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:54.202380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:54.202387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:54.204143 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:54.204499 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:54.204502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:54.204502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:54.204561 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:54.204716 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:54.204717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:54.204717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:54.204862 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:54.204976 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:54.205034 (+ 58us) server_negotiation.cc:299] Negotiation successful
0928 14:37:54.205100 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 28 14:37:54 dist-test-slave-wwwb krb5kdc[23106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606274, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260928 14:37:54.220852 23477 tablet.cc:2369] T 26de00009bb945228d12a97f068b5576 P 9db4549051ff4d3a979c7fd13b7dfaa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 28 14:37:54 dist-test-slave-wwwb krb5kdc[23106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606274, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.13.65@KRBTEST.COM
I20260928 14:37:54.228768 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:54.221721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35638 (local address 127.22.13.65:36845)
0928 14:37:54.221775 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:54.221777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:54.221871 (+ 94us) server_negotiation.cc:407] Connection header received
0928 14:37:54.221921 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:54.221922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:54.221931 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:54.222042 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:54.222614 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.223086 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.223782 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.223962 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.224013 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:54.226384 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:54.226385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:54.226385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:54.226392 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:54.227805 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:54.228113 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:54.228114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:54.228114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:54.228164 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:54.228318 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:54.228319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:54.228319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:54.228414 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:54.228485 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:54.228636 (+ 151us) server_negotiation.cc:299] Negotiation successful
0928 14:37:54.228700 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
Sep 28 14:37:54 dist-test-slave-wwwb krb5kdc[23106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 14:37:54 dist-test-slave-wwwb krb5kdc[23106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.65@KRBTEST.COM
I20260928 14:37:54.250689 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:54.243942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35654 (local address 127.22.13.65:36845)
0928 14:37:54.244023 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:54.244025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:54.244073 (+ 48us) server_negotiation.cc:407] Connection header received
0928 14:37:54.244138 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:54.244139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:54.244149 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:54.244199 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:54.244742 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.245213 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.246024 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:54.246207 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:54.246264 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:54.248560 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:54.248562 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:54.248562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:54.248569 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:54.249738 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:54.250078 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:54.250078 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:54.250079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:54.250128 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:54.250271 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:54.250271 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:54.250271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:54.250357 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:54.250422 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:54.250530 (+ 108us) server_negotiation.cc:299] Negotiation successful
0928 14:37:54.250587 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260928 14:37:54.251194 23256 tablet_service.cc:2908] Rejecting scan request for tablet 26de00009bb945228d12a97f068b5576: Uninitialized: clean time has not yet been initialized
I20260928 14:37:54.252110 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 23213
I20260928 14:37:54.256676 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 23347
I20260928 14:37:54.261313 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 23481
I20260928 14:37:54.265647 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 23119
2026-09-28T14:37:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1908 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:37:54 dist-test-slave-wwwb krb5kdc[23651](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:37:54 dist-test-slave-wwwb krb5kdc[23651](info): set up 2 sockets
Sep 28 14:37:54 dist-test-slave-wwwb krb5kdc[23651](info): commencing operation
krb5kdc: starting...
W20260928 14:37:55.289853 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:37:55 dist-test-slave-wwwb krb5kdc[23651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:37:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:37:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:37:55.409345 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41281
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:44217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:41281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:37:55.490818 23664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:37:55.491050 23664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:37:55.491074 23664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:37:55.492254 23664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:37:55.492297 23664 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 14:37:55.492311 23664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:37:55.492324 23664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:37:55.492337 23664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:37:55.492350 23664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:37:55.493820 23664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:41281
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:37:55.494118 23664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:37:55.494315 23664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:37:55.499225 23670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:55.499411 23672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:55.499229 23669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:55.499362 23664 server_base.cc:1034] running on GCE node
I20260928 14:37:55.499812 23664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:37:55.500186 23664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:37:55.501366 23664 hybrid_clock.cc:648] HybridClock initialized: now 1790606275501333 us; error 45 us; skew 500 ppm
Sep 28 14:37:55 dist-test-slave-wwwb krb5kdc[23651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606275, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:37:55.508550 23664 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:37:55.509112 23664 webserver.cc:460] Webserver started at http://127.22.13.126:42965/ using document root <none> and password file <none>
I20260928 14:37:55.509358 23664 fs_manager.cc:359] Metadata directory not provided
I20260928 14:37:55.509423 23664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:37:55.509567 23664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:37:55.510782 23664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "0563d78eac4f447686579df1b281f9aa"
format_stamp: "Formatted at 2026-09-28 14:37:55 on dist-test-slave-wwwb"
server_key: "f5d2440e7b2bd60220c1c599d3279e4b"
server_key_iv: "95e67378c67b124f432c28bc678198c1"
server_key_version: "encryptionkey@0"
I20260928 14:37:55.511164 23664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "0563d78eac4f447686579df1b281f9aa"
format_stamp: "Formatted at 2026-09-28 14:37:55 on dist-test-slave-wwwb"
server_key: "f5d2440e7b2bd60220c1c599d3279e4b"
server_key_iv: "95e67378c67b124f432c28bc678198c1"
server_key_version: "encryptionkey@0"
I20260928 14:37:55.512588 23664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:37:55.513324 23679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:55.513513 23664 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:37:55.513612 23664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "0563d78eac4f447686579df1b281f9aa"
format_stamp: "Formatted at 2026-09-28 14:37:55 on dist-test-slave-wwwb"
server_key: "f5d2440e7b2bd60220c1c599d3279e4b"
server_key_iv: "95e67378c67b124f432c28bc678198c1"
server_key_version: "encryptionkey@0"
I20260928 14:37:55.513697 23664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:37:55.527144 23664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:37:55.529080 23664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:37:55.529255 23664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:37:55.534214 23664 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:41281
I20260928 14:37:55.534268 23741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:41281 every 8 connection(s)
I20260928 14:37:55.534617 23664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:37:55.535233 23742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:55.537592 23742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa: Bootstrap starting.
I20260928 14:37:55.538321 23742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:55.538626 23742 log.cc:824] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa: Log is configured to *not* fsync() on all Append() calls
I20260928 14:37:55.539414 23742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa: No bootstrap required, opened a new log
I20260928 14:37:55.541656 23742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0563d78eac4f447686579df1b281f9aa" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41281 } }
I20260928 14:37:55.541821 23742 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:55.541883 23742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0563d78eac4f447686579df1b281f9aa, State: Initialized, Role: FOLLOWER
I20260928 14:37:55.542088 23742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0563d78eac4f447686579df1b281f9aa" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41281 } }
I20260928 14:37:55.542196 23742 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:37:55.542248 23742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:37:55.542300 23742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:55.542923 23742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0563d78eac4f447686579df1b281f9aa" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41281 } }
I20260928 14:37:55.543041 23742 leader_election.cc:302] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0563d78eac4f447686579df1b281f9aa; no voters:
I20260928 14:37:55.543246 23742 leader_election.cc:288] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:37:55.543278 23745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:37:55.543454 23745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [term 1 LEADER]: Becoming Leader. State: Replica: 0563d78eac4f447686579df1b281f9aa, State: Running, Role: LEADER
I20260928 14:37:55.543541 23742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:37:55.543567 23745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0563d78eac4f447686579df1b281f9aa" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41281 } }
I20260928 14:37:55.544072 23746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0563d78eac4f447686579df1b281f9aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0563d78eac4f447686579df1b281f9aa" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41281 } } }
I20260928 14:37:55.544107 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 23664
I20260928 14:37:55.544083 23747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0563d78eac4f447686579df1b281f9aa. Latest consensus state: current_term: 1 leader_uuid: "0563d78eac4f447686579df1b281f9aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0563d78eac4f447686579df1b281f9aa" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41281 } } }
I20260928 14:37:55.544188 23746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [sys.catalog]: This master's current role is: LEADER
I20260928 14:37:55.544198 23747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa [sys.catalog]: This master's current role is: LEADER
I20260928 14:37:55.544226 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:37:55.544414 22581 external_mini_cluster.cc:1428] Setting key dff86e245101fc280aebefb3f90db461
I20260928 14:37:55.545324 23754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:37:55.545837 23754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:37:55.547595 23754 catalog_manager.cc:1347] Generated new cluster ID: 7cd556f57aa74f7699e15445a6badaeb
I20260928 14:37:55.547655 23754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:37:55 dist-test-slave-wwwb krb5kdc[23651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:55.554847 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:55.544893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33172 (local address 127.22.13.126:41281)
0928 14:37:55.545127 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:55.545138 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:55.545177 (+ 39us) server_negotiation.cc:407] Connection header received
0928 14:37:55.545308 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:55.545314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:55.545396 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:55.545533 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:55.546473 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.547781 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.548542 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.548989 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.549104 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:55.551489 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:55.551493 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:55.551493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:55.551505 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:55.553180 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:55.553595 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:55.553598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:55.553599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:55.553706 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:55.553908 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:55.553909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:55.553909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:55.554215 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:55.554380 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:55.554485 (+ 105us) server_negotiation.cc:299] Negotiation successful
0928 14:37:55.554635 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260928 14:37:55.559676 23754 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:37:55.560269 23754 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:37:55.569772 23754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa: Generated new TSK 0
I20260928 14:37:55.570030 23754 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:37:55.606819 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:41281
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:44217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:37:55.706593 23768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:37:55.706815 23768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:37:55.706840 23768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:37:55.708103 23768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:37:55.708154 23768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:37:55.708176 23768 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:37:55.708194 23768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:37:55.709754 23768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.13.126:41281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:37:55.710026 23768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:37:55.710273 23768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:37:55.715258 23776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:55.715265 23774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:55.715277 23773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:55.715466 23768 server_base.cc:1034] running on GCE node
I20260928 14:37:55.715662 23768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:37:55.715921 23768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:37:55.717080 23768 hybrid_clock.cc:648] HybridClock initialized: now 1790606275717053 us; error 35 us; skew 500 ppm
Sep 28 14:37:55 dist-test-slave-wwwb krb5kdc[23651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606275, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:37:55.719656 23768 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:37:55.720052 23768 webserver.cc:460] Webserver started at http://127.22.13.65:44359/ using document root <none> and password file <none>
I20260928 14:37:55.720252 23768 fs_manager.cc:359] Metadata directory not provided
I20260928 14:37:55.720291 23768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:37:55.720384 23768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:37:55.721417 23768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "c12c9d707cf24a189e31a7b2defa6d9f"
format_stamp: "Formatted at 2026-09-28 14:37:55 on dist-test-slave-wwwb"
server_key: "9ac8f8103d5a712c9dfab0ac56e2016f"
server_key_iv: "702a1065ca9e4baf69f2b6e5e22be861"
server_key_version: "encryptionkey@0"
I20260928 14:37:55.721752 23768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "c12c9d707cf24a189e31a7b2defa6d9f"
format_stamp: "Formatted at 2026-09-28 14:37:55 on dist-test-slave-wwwb"
server_key: "9ac8f8103d5a712c9dfab0ac56e2016f"
server_key_iv: "702a1065ca9e4baf69f2b6e5e22be861"
server_key_version: "encryptionkey@0"
I20260928 14:37:55.723232 23768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:37:55.724077 23783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:55.724305 23768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:37:55.724395 23768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "c12c9d707cf24a189e31a7b2defa6d9f"
format_stamp: "Formatted at 2026-09-28 14:37:55 on dist-test-slave-wwwb"
server_key: "9ac8f8103d5a712c9dfab0ac56e2016f"
server_key_iv: "702a1065ca9e4baf69f2b6e5e22be861"
server_key_version: "encryptionkey@0"
I20260928 14:37:55.724472 23768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:37:55.742672 23768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:37:55.744908 23768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:37:55.745060 23768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:37:55.745606 23768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:37:55.745642 23768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:55.745668 23768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:37:55.745714 23768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:37:55 dist-test-slave-wwwb krb5kdc[23651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606275, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:55.753064 23768 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:40759
I20260928 14:37:55.753156 23898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:40759 every 8 connection(s)
I20260928 14:37:55.753507 23768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:37:55.753710 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 23768
I20260928 14:37:55.753806 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:37:55.754004 22581 external_mini_cluster.cc:1428] Setting key b0e2d23a17705b06b7d09a867cc82b45
I20260928 14:37:55.756973 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:55.746183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:59425 (local address 127.22.13.126:41281)
0928 14:37:55.746245 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:55.746247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:55.746509 (+ 262us) server_negotiation.cc:407] Connection header received
0928 14:37:55.746987 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:55.746989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:55.747004 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:55.747112 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:55.747950 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.748766 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.750520 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.750831 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.750912 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:55.753530 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:55.753531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:55.753531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:55.753542 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:55.755545 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:55.755953 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:55.755954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:55.755955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:55.756022 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:55.756319 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:55.756320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:55.756320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:55.756432 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:55.756490 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:55.756840 (+ 350us) server_negotiation.cc:299] Negotiation successful
0928 14:37:55.756910 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 14:37:55.757272 23792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:55.746197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41281 (local address 127.22.13.65:59425)
0928 14:37:55.746382 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:55.746417 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:55.746887 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:55.747183 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:55.747187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:55.747555 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:55.747854 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:55.747861 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.748864 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.748865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:55.750422 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:55.750425 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.750924 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.750925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:55.751065 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:55.751697 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:55.751706 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:55.753412 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:55.755638 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:55.755644 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:55.755646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:55.755879 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:55.756093 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:55.756094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:55.756094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:55.756252 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:55.756513 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:55.756519 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:55.756698 (+ 179us) client_negotiation.cc:769] Sending connection context
0928 14:37:55.756821 (+ 123us) client_negotiation.cc:240] Negotiation successful
0928 14:37:55.757004 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":105,"threads_started":1}
I20260928 14:37:55.757791 23899 heartbeater.cc:347] Connected to a master server at 127.22.13.126:41281
I20260928 14:37:55.757879 23899 heartbeater.cc:464] Registering TS with master...
I20260928 14:37:55.758136 23899 heartbeater.cc:511] Master 127.22.13.126:41281 requested a full tablet report, sending...
I20260928 14:37:55.758731 23696 ts_manager.cc:194] Registered new tserver with Master: c12c9d707cf24a189e31a7b2defa6d9f (127.22.13.65:40759)
I20260928 14:37:55.759663 23696 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:59425
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
I20260928 14:37:55.764860 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:55.759020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:32955 (local address 127.22.13.126:41281)
0928 14:37:55.759062 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:55.759064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:55.759250 (+ 186us) server_negotiation.cc:407] Connection header received
0928 14:37:55.759286 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:55.759287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:55.759299 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:55.759364 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:55.759707 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.760533 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.761159 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.761475 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.761555 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:55.762456 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:55.762457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:55.762458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:55.762466 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:55.763907 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:55.764245 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:55.764247 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:55.764247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:55.764303 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:55.764474 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:55.764475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:55.764475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:55.764572 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:55.764629 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:55.764734 (+ 105us) server_negotiation.cc:299] Negotiation successful
0928 14:37:55.764796 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:37:55.764865 23906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:55.758957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41281 (local address 127.22.13.65:32955)
0928 14:37:55.759173 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:55.759184 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:55.759245 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:55.759368 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:55.759369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:55.759568 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:55.759628 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:55.759630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.760611 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.760612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:55.761080 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:55.761081 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.761549 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.761549 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:55.761605 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:55.762090 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:55.762094 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:55.762368 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:55.763989 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:55.763989 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:55.763990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:55.764154 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:55.764360 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:55.764360 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:55.764361 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:55.764422 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:55.764630 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:55.764631 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:55.764670 (+ 39us) client_negotiation.cc:769] Sending connection context
0928 14:37:55.764720 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 14:37:55.764799 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":158,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:37:55.780709 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:41281
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:44217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:37:55.862636 23908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:37:55.862861 23908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:37:55.862890 23908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:37:55.864035 23908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:37:55.864073 23908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:37:55.864094 23908 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:37:55.864110 23908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:37:55.865535 23908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.13.126:41281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:37:55.865736 23908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:37:55.865922 23908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:37:55.870764 23913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:55.870771 23914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:55.870775 23916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:55.870913 23908 server_base.cc:1034] running on GCE node
I20260928 14:37:55.871270 23908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:37:55.871529 23908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:37:55.872685 23908 hybrid_clock.cc:648] HybridClock initialized: now 1790606275872673 us; error 38 us; skew 500 ppm
Sep 28 14:37:55 dist-test-slave-wwwb krb5kdc[23651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606275, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:37:55.875273 23908 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:37:55.875746 23908 webserver.cc:460] Webserver started at http://127.22.13.66:35283/ using document root <none> and password file <none>
I20260928 14:37:55.875983 23908 fs_manager.cc:359] Metadata directory not provided
I20260928 14:37:55.876049 23908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:37:55.876197 23908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:37:55.877272 23908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "6bf35196da0740e7b5900e1c7f842a53"
format_stamp: "Formatted at 2026-09-28 14:37:55 on dist-test-slave-wwwb"
server_key: "482899fcad6ca844270ebea1b687da69"
server_key_iv: "4aa99c76910a5f978ddc236f76f49330"
server_key_version: "encryptionkey@0"
I20260928 14:37:55.877627 23908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "6bf35196da0740e7b5900e1c7f842a53"
format_stamp: "Formatted at 2026-09-28 14:37:55 on dist-test-slave-wwwb"
server_key: "482899fcad6ca844270ebea1b687da69"
server_key_iv: "4aa99c76910a5f978ddc236f76f49330"
server_key_version: "encryptionkey@0"
I20260928 14:37:55.879065 23908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:37:55.879787 23923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:55.879978 23908 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:55.880089 23908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "6bf35196da0740e7b5900e1c7f842a53"
format_stamp: "Formatted at 2026-09-28 14:37:55 on dist-test-slave-wwwb"
server_key: "482899fcad6ca844270ebea1b687da69"
server_key_iv: "4aa99c76910a5f978ddc236f76f49330"
server_key_version: "encryptionkey@0"
I20260928 14:37:55.880158 23908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:37:55.893292 23908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:37:55.895313 23908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:37:55.895481 23908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:37:55.896085 23908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:37:55.896134 23908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:55.896171 23908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:37:55.896189 23908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:37:55 dist-test-slave-wwwb krb5kdc[23651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606275, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:55.903357 23908 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:39683
I20260928 14:37:55.903443 24038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:39683 every 8 connection(s)
I20260928 14:37:55.903759 23908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:37:55.905526 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 23908
I20260928 14:37:55.905638 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:37:55.905823 22581 external_mini_cluster.cc:1428] Setting key 6202b3d68746826e0d24948b9cadf043
I20260928 14:37:55.906273 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:55.896489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:49485 (local address 127.22.13.126:41281)
0928 14:37:55.896556 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:55.896559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:55.896763 (+ 204us) server_negotiation.cc:407] Connection header received
0928 14:37:55.897248 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:55.897250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:55.897265 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:55.897353 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:55.898313 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.899158 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.900694 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.900905 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.900992 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:55.903359 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:55.903361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:55.903361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:55.903371 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:55.904823 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:55.905199 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:55.905200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:55.905200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:55.905252 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:55.905526 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:55.905526 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:55.905526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:55.905655 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:55.905719 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:55.906097 (+ 378us) server_negotiation.cc:299] Negotiation successful
0928 14:37:55.906187 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 14:37:55.906414 23932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:55.896485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41281 (local address 127.22.13.66:49485)
0928 14:37:55.896630 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:55.896672 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:55.897154 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:55.897442 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:55.897447 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:55.897837 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:55.898199 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:55.898207 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.899308 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.899309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:55.900604 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:55.900608 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.901016 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.901017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:55.901151 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:55.901705 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:55.901715 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:55.903256 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:55.904896 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:55.904901 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:55.904903 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:55.905133 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:55.905309 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:55.905309 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:55.905309 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:55.905461 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:55.905712 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:55.905718 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:55.905897 (+ 179us) client_negotiation.cc:769] Sending connection context
0928 14:37:55.906065 (+ 168us) client_negotiation.cc:240] Negotiation successful
0928 14:37:55.906202 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260928 14:37:55.907045 24039 heartbeater.cc:347] Connected to a master server at 127.22.13.126:41281
I20260928 14:37:55.907147 24039 heartbeater.cc:464] Registering TS with master...
I20260928 14:37:55.907326 24039 heartbeater.cc:511] Master 127.22.13.126:41281 requested a full tablet report, sending...
I20260928 14:37:55.908119 23695 ts_manager.cc:194] Registered new tserver with Master: 6bf35196da0740e7b5900e1c7f842a53 (127.22.13.66:39683)
I20260928 14:37:55.908634 23695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:49485
I20260928 14:37:55.914834 24046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:55.908224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41281 (local address 127.22.13.66:41655)
0928 14:37:55.908380 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:55.908392 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:55.908447 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:55.908907 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:55.908908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:55.909128 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:55.909196 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:55.909199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.909862 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.909863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:55.910841 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:55.910846 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.911219 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.911220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:55.911303 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:55.911792 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:55.911796 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:55.912098 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:55.913670 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:55.913671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:55.913672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:55.913866 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:55.914100 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:55.914101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:55.914102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:55.914163 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:55.914468 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:55.914469 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:55.914526 (+ 57us) client_negotiation.cc:769] Sending connection context
0928 14:37:55.914608 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 14:37:55.914718 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260928 14:37:55.914822 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:55.908773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:41655 (local address 127.22.13.126:41281)
0928 14:37:55.908813 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:55.908815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:55.908822 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:37:55.908842 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:55.908842 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:55.908851 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:55.908901 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:55.909277 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.909778 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.910961 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:55.911144 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:55.911199 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:55.912216 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:55.912217 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:55.912217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:55.912225 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:55.913585 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:55.913952 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:55.913954 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:55.913954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:55.914020 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:55.914238 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:55.914239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:55.914239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:55.914402 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:55.914478 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:55.914633 (+ 155us) server_negotiation.cc:299] Negotiation successful
0928 14:37:55.914717 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:37:55.932255 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:41281
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:44217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:37:56.014385 24048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:37:56.014647 24048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:37:56.014690 24048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:37:56.015929 24048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:37:56.015997 24048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:37:56.016032 24048 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:37:56.016069 24048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:37:56.018043 24048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.13.126:41281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:37:56.018318 24048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:37:56.018565 24048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:37:56.023489 24053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:56.023528 24056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:56.023504 24054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:56.023552 24048 server_base.cc:1034] running on GCE node
I20260928 14:37:56.023942 24048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:37:56.024224 24048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:37:56.025363 24048 hybrid_clock.cc:648] HybridClock initialized: now 1790606276025353 us; error 27 us; skew 500 ppm
Sep 28 14:37:56 dist-test-slave-wwwb krb5kdc[23651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606276, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:37:56.027992 24048 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:37:56.028477 24048 webserver.cc:460] Webserver started at http://127.22.13.67:33665/ using document root <none> and password file <none>
I20260928 14:37:56.028702 24048 fs_manager.cc:359] Metadata directory not provided
I20260928 14:37:56.028767 24048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:37:56.028914 24048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:37:56.030087 24048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "5fa3b206cad4423087e0646de8008b85"
format_stamp: "Formatted at 2026-09-28 14:37:56 on dist-test-slave-wwwb"
server_key: "6b00806f487996d943c93602d08be563"
server_key_iv: "ac1d1ce7bac0c128fcd590e77824e5c3"
server_key_version: "encryptionkey@0"
I20260928 14:37:56.030448 24048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "5fa3b206cad4423087e0646de8008b85"
format_stamp: "Formatted at 2026-09-28 14:37:56 on dist-test-slave-wwwb"
server_key: "6b00806f487996d943c93602d08be563"
server_key_iv: "ac1d1ce7bac0c128fcd590e77824e5c3"
server_key_version: "encryptionkey@0"
I20260928 14:37:56.031793 24048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:37:56.032516 24063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:56.032677 24048 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:37:56.032778 24048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "5fa3b206cad4423087e0646de8008b85"
format_stamp: "Formatted at 2026-09-28 14:37:56 on dist-test-slave-wwwb"
server_key: "6b00806f487996d943c93602d08be563"
server_key_iv: "ac1d1ce7bac0c128fcd590e77824e5c3"
server_key_version: "encryptionkey@0"
I20260928 14:37:56.032852 24048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:37:56.045173 24048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:37:56.047626 24048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:37:56.047791 24048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:37:56.048323 24048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:37:56.048383 24048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:56.048429 24048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:37:56.048473 24048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:56.055148 24048 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:46677
I20260928 14:37:56.055217 24178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:46677 every 8 connection(s)
I20260928 14:37:56.055496 24048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
Sep 28 14:37:56 dist-test-slave-wwwb krb5kdc[23651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606276, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:56.057512 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 24048
I20260928 14:37:56.057593 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:37:56.057775 22581 external_mini_cluster.cc:1428] Setting key 412aaa456253bcf369e31c28faa1cf49
I20260928 14:37:56.059288 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.048804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:41809 (local address 127.22.13.126:41281)
0928 14:37:56.048851 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.048854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.049090 (+ 236us) server_negotiation.cc:407] Connection header received
0928 14:37:56.049624 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.049625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.049639 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.049693 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:56.050718 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.051580 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.053178 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.053429 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.053521 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.056121 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:56.056123 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:56.056123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:56.056130 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:56.057695 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:56.058234 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:56.058235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:56.058235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:56.058307 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:56.058610 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:56.058611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:56.058611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:56.058735 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:56.058807 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.059102 (+ 295us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.059201 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:37:56.059471 24072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.048809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41281 (local address 127.22.13.67:41809)
0928 14:37:56.048966 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:56.049003 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:56.049514 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:56.049781 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:56.049785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:56.050222 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:56.050569 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.050577 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.051682 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.051683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.053091 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.053094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.053532 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.053533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.053695 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.054374 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:56.054384 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:56.056028 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:56.057773 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:56.057780 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:56.057782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:56.058137 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:56.058392 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:56.058393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:56.058393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:56.058531 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:56.058798 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:56.058804 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:56.058963 (+ 159us) client_negotiation.cc:769] Sending connection context
0928 14:37:56.059065 (+ 102us) client_negotiation.cc:240] Negotiation successful
0928 14:37:56.059231 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260928 14:37:56.059875 24179 heartbeater.cc:347] Connected to a master server at 127.22.13.126:41281
I20260928 14:37:56.059988 24179 heartbeater.cc:464] Registering TS with master...
I20260928 14:37:56.060168 24179 heartbeater.cc:511] Master 127.22.13.126:41281 requested a full tablet report, sending...
I20260928 14:37:56.060523 23695 ts_manager.cc:194] Registered new tserver with Master: 5fa3b206cad4423087e0646de8008b85 (127.22.13.67:46677)
I20260928 14:37:56.060904 23695 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:41809
I20260928 14:37:56.066208 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.060986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:57903 (local address 127.22.13.126:41281)
0928 14:37:56.061028 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.061030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.061118 (+ 88us) server_negotiation.cc:407] Connection header received
0928 14:37:56.061216 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.061217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.061228 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.061284 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:56.061624 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.062155 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.062793 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.062978 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.063028 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.063858 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:56.063859 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:56.063859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:56.063866 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:56.065206 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:56.065531 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:56.065532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:56.065532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:56.065582 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:56.065739 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:56.065739 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:56.065740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:56.065840 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:56.065889 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.066077 (+ 188us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.066140 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:37:56.066223 24185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.060920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41281 (local address 127.22.13.67:57903)
0928 14:37:56.061046 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:56.061057 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:56.061131 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:56.061288 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:56.061288 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:56.061491 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:56.061550 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.061552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.062238 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.062239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.062725 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.062726 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.063039 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.063040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.063093 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.063498 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:56.063502 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:56.063780 (+ 278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:56.065289 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:56.065290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:56.065290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:56.065464 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:56.065636 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:56.065637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:56.065637 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:56.065690 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:56.065889 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:56.065890 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:56.065929 (+ 39us) client_negotiation.cc:769] Sending connection context
0928 14:37:56.066056 (+ 127us) client_negotiation.cc:240] Negotiation successful
0928 14:37:56.066142 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":74,"threads_started":1}
I20260928 14:37:56.069679 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 14:37:56 dist-test-slave-wwwb krb5kdc[23651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606276, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 14:37:56 dist-test-slave-wwwb krb5kdc[23651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606276, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:56.092355 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.085699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33182 (local address 127.22.13.126:41281)
0928 14:37:56.085761 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.085763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.085772 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:37:56.085797 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.085798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.085808 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.085864 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:56.086464 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.086979 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.087716 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.087909 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.087963 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.090218 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:56.090219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:56.090220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:56.090227 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:56.091367 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:56.091679 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:56.091680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:56.091680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:56.091727 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:56.091877 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:56.091878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:56.091878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:56.091978 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:56.092051 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.092136 (+ 85us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.092200 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 14:37:56.094080 23695 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33182:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:37:56.095603 23695 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:37:56.102274 24196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.097997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:40759 (local address 127.0.0.1:50044)
0928 14:37:56.098286 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:56.098334 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:56.098422 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:56.098759 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:56.098760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:56.098763 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:56.098813 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.098814 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.099746 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.099747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.100985 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.100987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.101910 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.101911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.102040 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.102060 (+ 20us) client_negotiation.cc:769] Sending connection context
0928 14:37:56.102127 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 14:37:56.102188 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":3,"thread_start_us":57,"threads_started":1}
I20260928 14:37:56.102368 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.098270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:39683 (local address 127.0.0.1:37592)
0928 14:37:56.098443 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:56.098452 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:56.098529 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:56.098845 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:56.098845 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:56.098847 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:56.098882 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.098884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.100163 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.100165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.101180 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.101182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.102059 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.102060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.102126 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.102131 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:37:56.102195 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 14:37:56.102246 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":3,"thread_start_us":67,"threads_started":1}
I20260928 14:37:56.102566 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.098109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50044 (local address 127.22.13.65:40759)
0928 14:37:56.098446 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.098451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.098466 (+ 15us) server_negotiation.cc:407] Connection header received
0928 14:37:56.098507 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.098509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.098523 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.098610 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:56.098908 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.099645 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.101065 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.101821 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.101901 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.101996 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.102427 (+ 431us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.102472 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":76,"threads_started":1}
I20260928 14:37:56.102890 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.098135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37592 (local address 127.22.13.66:39683)
0928 14:37:56.098271 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.098276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.098503 (+ 227us) server_negotiation.cc:407] Connection header received
0928 14:37:56.098693 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.098696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.098712 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.098784 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:56.098981 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.100059 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.101253 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.101994 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.102060 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.102111 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.102209 (+ 98us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.102246 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":67,"threads_started":1}
I20260928 14:37:56.103297 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.098007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:46677 (local address 127.0.0.1:50254)
0928 14:37:56.098386 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:56.098396 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:56.098460 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:56.098734 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:56.098737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:56.098739 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:56.098813 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.098815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.101248 (+ 2433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.101249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.101937 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.101938 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.103059 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.103060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.103139 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.103143 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:37:56.103194 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 14:37:56.103234 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":97,"threads_started":1}
I20260928 14:37:56.103549 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.098247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50254 (local address 127.22.13.67:46677)
0928 14:37:56.098399 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.098403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.098537 (+ 134us) server_negotiation.cc:407] Connection header received
0928 14:37:56.098579 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.098581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.098598 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.098670 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:56.098895 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.100065 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.102060 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.102818 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.102912 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.102969 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.103407 (+ 438us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.103450 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260928 14:37:56.103480 23833 tablet_service.cc:1467] Processing CreateTablet for tablet 0edeff22577548ceb03008013cd0cf11 (DEFAULT_TABLE table=test-table [id=d5778aa1e1864a1d83f02b26e66d232e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:37:56.103539 23973 tablet_service.cc:1467] Processing CreateTablet for tablet 0edeff22577548ceb03008013cd0cf11 (DEFAULT_TABLE table=test-table [id=d5778aa1e1864a1d83f02b26e66d232e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:37:56.103832 23833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0edeff22577548ceb03008013cd0cf11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:56.103843 23973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0edeff22577548ceb03008013cd0cf11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:56.106753 24113 tablet_service.cc:1467] Processing CreateTablet for tablet 0edeff22577548ceb03008013cd0cf11 (DEFAULT_TABLE table=test-table [id=d5778aa1e1864a1d83f02b26e66d232e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:37:56.107128 24202 tablet_bootstrap.cc:492] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53: Bootstrap starting.
I20260928 14:37:56.107159 24113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0edeff22577548ceb03008013cd0cf11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:56.107715 24202 tablet_bootstrap.cc:654] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:56.108031 24202 log.cc:824] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53: Log is configured to *not* fsync() on all Append() calls
I20260928 14:37:56.108268 24203 tablet_bootstrap.cc:492] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f: Bootstrap starting.
I20260928 14:37:56.108906 24202 tablet_bootstrap.cc:492] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53: No bootstrap required, opened a new log
I20260928 14:37:56.108942 24203 tablet_bootstrap.cc:654] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:56.109027 24202 ts_tablet_manager.cc:1397] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:37:56.109261 24203 log.cc:824] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f: Log is configured to *not* fsync() on all Append() calls
I20260928 14:37:56.110574 24206 tablet_bootstrap.cc:492] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85: Bootstrap starting.
I20260928 14:37:56.111292 24203 tablet_bootstrap.cc:492] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f: No bootstrap required, opened a new log
I20260928 14:37:56.111387 24203 ts_tablet_manager.cc:1397] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260928 14:37:56.111399 24202 raft_consensus.cc:348] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } }
I20260928 14:37:56.111595 24206 tablet_bootstrap.cc:654] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:56.111596 24202 raft_consensus.cc:374] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:56.111637 24202 raft_consensus.cc:729] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bf35196da0740e7b5900e1c7f842a53, State: Initialized, Role: FOLLOWER
I20260928 14:37:56.111795 24202 consensus_queue.cc:260] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } }
I20260928 14:37:56.111966 24206 log.cc:824] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85: Log is configured to *not* fsync() on all Append() calls
I20260928 14:37:56.112083 24202 ts_tablet_manager.cc:1428] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:37:56.112224 24039 heartbeater.cc:503] Master 127.22.13.126:41281 was elected leader, sending a full tablet report...
I20260928 14:37:56.113346 24203 raft_consensus.cc:348] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } }
I20260928 14:37:56.113548 24203 raft_consensus.cc:374] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:56.113595 24203 raft_consensus.cc:729] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c12c9d707cf24a189e31a7b2defa6d9f, State: Initialized, Role: FOLLOWER
I20260928 14:37:56.113718 24203 consensus_queue.cc:260] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } }
I20260928 14:37:56.114030 24203 ts_tablet_manager.cc:1428] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:37:56.114044 23899 heartbeater.cc:503] Master 127.22.13.126:41281 was elected leader, sending a full tablet report...
I20260928 14:37:56.114667 24206 tablet_bootstrap.cc:492] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85: No bootstrap required, opened a new log
I20260928 14:37:56.114755 24206 ts_tablet_manager.cc:1397] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260928 14:37:56.116291 24206 raft_consensus.cc:348] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } }
I20260928 14:37:56.116451 24206 raft_consensus.cc:374] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:56.116503 24206 raft_consensus.cc:729] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa3b206cad4423087e0646de8008b85, State: Initialized, Role: FOLLOWER
I20260928 14:37:56.116634 24206 consensus_queue.cc:260] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } }
I20260928 14:37:56.116899 24206 ts_tablet_manager.cc:1428] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:37:56.116947 24179 heartbeater.cc:503] Master 127.22.13.126:41281 was elected leader, sending a full tablet report...
W20260928 14:37:56.154354 24040 tablet.cc:2369] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:37:56.254372 23900 tablet.cc:2369] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:37:56.306142 24180 tablet.cc:2369] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:37:56.383965 24210 raft_consensus.cc:482] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:37:56.384097 24210 raft_consensus.cc:504] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } }
I20260928 14:37:56.384390 24210 leader_election.cc:288] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c12c9d707cf24a189e31a7b2defa6d9f (127.22.13.65:40759), 6bf35196da0740e7b5900e1c7f842a53 (127.22.13.66:39683)
I20260928 14:37:56.388514 24072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.384595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:39683 (local address 127.22.13.67:60441)
0928 14:37:56.384657 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:56.384669 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:56.384713 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:56.384881 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:56.384882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:56.384885 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:56.384963 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.384964 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.386059 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.386059 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.387439 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.387442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.388217 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.388218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.388312 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.388317 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:37:56.388373 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 14:37:56.388415 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":39}
I20260928 14:37:56.388597 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.384658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:60441 (local address 127.22.13.66:39683)
0928 14:37:56.384719 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.384721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.384735 (+ 14us) server_negotiation.cc:407] Connection header received
0928 14:37:56.384792 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.384793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.384802 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.384856 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:56.385062 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.385883 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.387558 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.388126 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.388200 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.388220 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.388427 (+ 207us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.388503 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:37:56.389034 24211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.384595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:40759 (local address 127.22.13.67:42031)
0928 14:37:56.384813 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:56.384820 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:56.384881 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:56.385028 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:56.385029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:56.385031 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:56.385094 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.385095 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.386048 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.386049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.387439 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.387442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.388217 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.388218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.388805 (+ 587us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.388808 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:37:56.388871 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 14:37:56.388929 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":167,"thread_start_us":99,"threads_started":1}
I20260928 14:37:56.389108 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.384665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:42031 (local address 127.22.13.65:40759)
0928 14:37:56.384710 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.384711 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.384921 (+ 210us) server_negotiation.cc:407] Connection header received
0928 14:37:56.384955 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.384957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.384964 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.385003 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:56.385158 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.385897 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.387553 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.388116 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.388200 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.388220 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.388908 (+ 688us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.389010 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:37:56.389560 23853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0edeff22577548ceb03008013cd0cf11" candidate_uuid: "5fa3b206cad4423087e0646de8008b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" is_pre_election: true
I20260928 14:37:56.389752 23853 raft_consensus.cc:2454] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fa3b206cad4423087e0646de8008b85 in term 0.
I20260928 14:37:56.389756 23993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0edeff22577548ceb03008013cd0cf11" candidate_uuid: "5fa3b206cad4423087e0646de8008b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bf35196da0740e7b5900e1c7f842a53" is_pre_election: true
I20260928 14:37:56.389897 23993 raft_consensus.cc:2454] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fa3b206cad4423087e0646de8008b85 in term 0.
I20260928 14:37:56.390141 24066 leader_election.cc:302] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa3b206cad4423087e0646de8008b85, c12c9d707cf24a189e31a7b2defa6d9f; no voters:
I20260928 14:37:56.390527 24210 raft_consensus.cc:2781] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:37:56.390590 24210 raft_consensus.cc:482] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:37:56.390614 24210 raft_consensus.cc:3037] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:56.391564 24210 raft_consensus.cc:504] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } }
I20260928 14:37:56.391737 24210 leader_election.cc:288] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [CANDIDATE]: Term 1 election: Requested vote from peers c12c9d707cf24a189e31a7b2defa6d9f (127.22.13.65:40759), 6bf35196da0740e7b5900e1c7f842a53 (127.22.13.66:39683)
I20260928 14:37:56.391884 23993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0edeff22577548ceb03008013cd0cf11" candidate_uuid: "5fa3b206cad4423087e0646de8008b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bf35196da0740e7b5900e1c7f842a53"
I20260928 14:37:56.391914 23853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0edeff22577548ceb03008013cd0cf11" candidate_uuid: "5fa3b206cad4423087e0646de8008b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12c9d707cf24a189e31a7b2defa6d9f"
I20260928 14:37:56.391983 23993 raft_consensus.cc:3037] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:56.391985 23853 raft_consensus.cc:3037] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:56.392674 23853 raft_consensus.cc:2454] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fa3b206cad4423087e0646de8008b85 in term 1.
I20260928 14:37:56.392683 23993 raft_consensus.cc:2454] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fa3b206cad4423087e0646de8008b85 in term 1.
I20260928 14:37:56.392836 24066 leader_election.cc:302] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa3b206cad4423087e0646de8008b85, c12c9d707cf24a189e31a7b2defa6d9f; no voters:
I20260928 14:37:56.392944 24210 raft_consensus.cc:2781] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:37:56.393128 24210 raft_consensus.cc:686] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [term 1 LEADER]: Becoming Leader. State: Replica: 5fa3b206cad4423087e0646de8008b85, State: Running, Role: LEADER
I20260928 14:37:56.393229 24210 consensus_queue.cc:237] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } }
I20260928 14:37:56.394090 23696 catalog_manager.cc:5495] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fa3b206cad4423087e0646de8008b85 (127.22.13.67). New cstate: current_term: 1 leader_uuid: "5fa3b206cad4423087e0646de8008b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } health_report { overall_health: UNKNOWN } } }
I20260928 14:37:56.456732 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.454168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50268 (local address 127.22.13.67:46677)
0928 14:37:56.454233 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.454235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.454244 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:37:56.454287 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.454288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.454304 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.454367 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:37:56.454526 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.454998 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.455818 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.456011 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.456104 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.456324 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:37:56.456430 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:37:56.456540 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.456593 (+ 53us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.456616 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:37:56.458606 23993 raft_consensus.cc:1261] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53 [term 1 FOLLOWER]: Refusing update from remote peer 5fa3b206cad4423087e0646de8008b85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:37:56.458642 23853 raft_consensus.cc:1261] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f [term 1 FOLLOWER]: Refusing update from remote peer 5fa3b206cad4423087e0646de8008b85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:37:56.459002 24212 consensus_queue.cc:1035] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:37:56.459126 24210 consensus_queue.cc:1035] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:37:56.461998 24217 mvcc.cc:205] Tried to move back new op lower bound from 7334323308371824640 to 7334323308107227136. Current Snapshot: MvccSnapshot[applied={T|T < 7334323308371824640}]
I20260928 14:37:56.462182 24215 mvcc.cc:205] Tried to move back new op lower bound from 7334323308371824640 to 7334323308107227136. Current Snapshot: MvccSnapshot[applied={T|T < 7334323308371824640 or (T in {7334323308371824640})}]
I20260928 14:37:56.464957 23696 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:37:56.465070 23696 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:37:56.465888 23696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa: Sending DeleteTablet for 3 replicas of tablet 0edeff22577548ceb03008013cd0cf11
I20260928 14:37:56.466306 24113 tablet_service.cc:1514] Processing DeleteTablet for tablet 0edeff22577548ceb03008013cd0cf11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:37:56 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:50254
I20260928 14:37:56.466318 23833 tablet_service.cc:1514] Processing DeleteTablet for tablet 0edeff22577548ceb03008013cd0cf11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:37:56 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:50044
I20260928 14:37:56.466536 23973 tablet_service.cc:1514] Processing DeleteTablet for tablet 0edeff22577548ceb03008013cd0cf11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:37:56 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:37592
I20260928 14:37:56.466571 23696 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33182:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:37:56.466717 23696 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:37:56.466818 24227 tablet_replica.cc:330] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53: stopping tablet replica
I20260928 14:37:56.466951 24227 raft_consensus.cc:2229] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:37:56.467031 24226 tablet_replica.cc:330] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85: stopping tablet replica
I20260928 14:37:56.467127 24226 raft_consensus.cc:2229] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:37:56.467221 24226 raft_consensus.cc:2258] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:37:56.467226 24227 raft_consensus.cc:2258] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:37:56.467708 24226 ts_tablet_manager.cc:1905] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:37:56.468091 23833 tablet_service.cc:1467] Processing CreateTablet for tablet 21e5aada8e6a47bc8d49c18052d17258 (DEFAULT_TABLE table=test-table [id=8c458cf1f008464db91e6137f5ba9e25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:37:56.468181 23973 tablet_service.cc:1467] Processing CreateTablet for tablet 21e5aada8e6a47bc8d49c18052d17258 (DEFAULT_TABLE table=test-table [id=8c458cf1f008464db91e6137f5ba9e25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:37:56.468219 23833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21e5aada8e6a47bc8d49c18052d17258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:56.468299 23973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21e5aada8e6a47bc8d49c18052d17258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:56.469465 24225 tablet_replica.cc:330] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f: stopping tablet replica
I20260928 14:37:56.469583 24225 raft_consensus.cc:2229] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:37:56.469626 24203 tablet_bootstrap.cc:492] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f: Bootstrap starting.
I20260928 14:37:56.469643 24202 tablet_bootstrap.cc:492] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53: Bootstrap starting.
I20260928 14:37:56.469688 24225 raft_consensus.cc:2258] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:37:56.470008 24203 tablet_bootstrap.cc:654] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:56.470074 24202 tablet_bootstrap.cc:654] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:56.470183 24113 tablet_service.cc:1467] Processing CreateTablet for tablet 21e5aada8e6a47bc8d49c18052d17258 (DEFAULT_TABLE table=test-table [id=8c458cf1f008464db91e6137f5ba9e25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:37:56.470530 24113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21e5aada8e6a47bc8d49c18052d17258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:56.470997 24227 ts_tablet_manager.cc:1905] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:37:56.471827 24206 tablet_bootstrap.cc:492] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85: Bootstrap starting.
I20260928 14:37:56.471969 24226 ts_tablet_manager.cc:1918] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:37:56.472054 24226 log.cc:1196] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/0edeff22577548ceb03008013cd0cf11
I20260928 14:37:56.472265 24206 tablet_bootstrap.cc:654] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:56.472312 24226 ts_tablet_manager.cc:1939] T 0edeff22577548ceb03008013cd0cf11 P 5fa3b206cad4423087e0646de8008b85: Deleting consensus metadata
I20260928 14:37:56.472712 23680 catalog_manager.cc:4841] TS 5fa3b206cad4423087e0646de8008b85 (127.22.13.67:46677): tablet 0edeff22577548ceb03008013cd0cf11 (table test-table [id=d5778aa1e1864a1d83f02b26e66d232e]) successfully deleted
I20260928 14:37:56.472909 24227 ts_tablet_manager.cc:1918] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:37:56.472985 24227 log.cc:1196] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/0edeff22577548ceb03008013cd0cf11
I20260928 14:37:56.473238 24227 ts_tablet_manager.cc:1939] T 0edeff22577548ceb03008013cd0cf11 P 6bf35196da0740e7b5900e1c7f842a53: Deleting consensus metadata
I20260928 14:37:56.473304 24225 ts_tablet_manager.cc:1905] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:37:56.473695 23683 catalog_manager.cc:4841] TS 6bf35196da0740e7b5900e1c7f842a53 (127.22.13.66:39683): tablet 0edeff22577548ceb03008013cd0cf11 (table test-table [id=d5778aa1e1864a1d83f02b26e66d232e]) successfully deleted
I20260928 14:37:56.473909 24203 tablet_bootstrap.cc:492] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f: No bootstrap required, opened a new log
I20260928 14:37:56.473987 24203 ts_tablet_manager.cc:1397] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260928 14:37:56.474159 24203 raft_consensus.cc:348] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.474253 24203 raft_consensus.cc:374] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:56.474280 24203 raft_consensus.cc:729] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c12c9d707cf24a189e31a7b2defa6d9f, State: Initialized, Role: FOLLOWER
I20260928 14:37:56.474341 24203 consensus_queue.cc:260] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.474481 24203 ts_tablet_manager.cc:1428] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:56.475032 24225 ts_tablet_manager.cc:1918] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:37:56.475085 24206 tablet_bootstrap.cc:492] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85: No bootstrap required, opened a new log
I20260928 14:37:56.475096 24225 log.cc:1196] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/wal/wals/0edeff22577548ceb03008013cd0cf11
I20260928 14:37:56.475133 24206 ts_tablet_manager.cc:1397] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260928 14:37:56.475291 24206 raft_consensus.cc:348] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.475368 24206 raft_consensus.cc:374] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:56.475394 24206 raft_consensus.cc:729] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa3b206cad4423087e0646de8008b85, State: Initialized, Role: FOLLOWER
I20260928 14:37:56.475447 24206 consensus_queue.cc:260] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.475565 24206 ts_tablet_manager.cc:1428] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:56.475709 24225 ts_tablet_manager.cc:1939] T 0edeff22577548ceb03008013cd0cf11 P c12c9d707cf24a189e31a7b2defa6d9f: Deleting consensus metadata
I20260928 14:37:56.476104 23682 catalog_manager.cc:4841] TS c12c9d707cf24a189e31a7b2defa6d9f (127.22.13.65:40759): tablet 0edeff22577548ceb03008013cd0cf11 (table test-table [id=d5778aa1e1864a1d83f02b26e66d232e]) successfully deleted
I20260928 14:37:56.476219 24202 tablet_bootstrap.cc:492] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53: No bootstrap required, opened a new log
I20260928 14:37:56.476270 24202 ts_tablet_manager.cc:1397] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260928 14:37:56.476620 24202 raft_consensus.cc:348] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.476681 24202 raft_consensus.cc:374] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:56.476696 24202 raft_consensus.cc:729] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bf35196da0740e7b5900e1c7f842a53, State: Initialized, Role: FOLLOWER
I20260928 14:37:56.476751 24202 consensus_queue.cc:260] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.476861 24202 ts_tablet_manager.cc:1428] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260928 14:37:56.541361 24207 raft_consensus.cc:482] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:37:56.541457 24207 raft_consensus.cc:504] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.541821 24207 leader_election.cc:288] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c12c9d707cf24a189e31a7b2defa6d9f (127.22.13.65:40759), 5fa3b206cad4423087e0646de8008b85 (127.22.13.67:46677)
I20260928 14:37:56.545976 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.542067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:46677 (local address 127.22.13.66:37319)
0928 14:37:56.542220 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:56.542249 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:56.542317 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:56.542589 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:56.542590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:56.542592 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:56.542772 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.542775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.543708 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.543709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.544841 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.544843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.545644 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.545644 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.545725 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.545727 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:37:56.545790 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 14:37:56.545841 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":14,"thread_start_us":89,"threads_started":1}
I20260928 14:37:56.545984 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.542113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:38821 (local address 127.22.13.65:40759)
0928 14:37:56.542173 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.542175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.542310 (+ 135us) server_negotiation.cc:407] Connection header received
0928 14:37:56.542429 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.542431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.542443 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.542516 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:56.542684 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.543295 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.544948 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.545495 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.545546 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.545554 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.545830 (+ 276us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.545880 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260928 14:37:56.546020 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.542245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:37319 (local address 127.22.13.67:46677)
0928 14:37:56.542292 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.542294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.542305 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:37:56.542445 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.542446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.542457 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.542530 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:56.542872 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.543585 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.544944 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.545499 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.545546 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.545554 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.545889 (+ 335us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.545928 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:37:56.546195 24232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.542064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:40759 (local address 127.22.13.66:38821)
0928 14:37:56.542220 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:56.542249 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:56.542333 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:56.542527 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:56.542528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:56.542530 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:56.542610 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.542612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.543393 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.543394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.544829 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.544833 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.545643 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.545643 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.545718 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.545723 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:37:56.545806 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 14:37:56.545841 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":4,"thread_start_us":60,"threads_started":1}
I20260928 14:37:56.546296 24133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21e5aada8e6a47bc8d49c18052d17258" candidate_uuid: "6bf35196da0740e7b5900e1c7f842a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa3b206cad4423087e0646de8008b85" is_pre_election: true
I20260928 14:37:56.546437 24133 raft_consensus.cc:2454] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bf35196da0740e7b5900e1c7f842a53 in term 0.
I20260928 14:37:56.546586 23853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21e5aada8e6a47bc8d49c18052d17258" candidate_uuid: "6bf35196da0740e7b5900e1c7f842a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" is_pre_election: true
I20260928 14:37:56.546679 23853 raft_consensus.cc:2454] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bf35196da0740e7b5900e1c7f842a53 in term 0.
I20260928 14:37:56.546643 23924 leader_election.cc:302] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa3b206cad4423087e0646de8008b85, 6bf35196da0740e7b5900e1c7f842a53; no voters:
I20260928 14:37:56.546777 24207 raft_consensus.cc:2781] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:37:56.546831 24207 raft_consensus.cc:482] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:37:56.546852 24207 raft_consensus.cc:3037] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:56.547557 24207 raft_consensus.cc:504] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.547708 24207 leader_election.cc:288] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [CANDIDATE]: Term 1 election: Requested vote from peers c12c9d707cf24a189e31a7b2defa6d9f (127.22.13.65:40759), 5fa3b206cad4423087e0646de8008b85 (127.22.13.67:46677)
I20260928 14:37:56.547851 23853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21e5aada8e6a47bc8d49c18052d17258" candidate_uuid: "6bf35196da0740e7b5900e1c7f842a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12c9d707cf24a189e31a7b2defa6d9f"
I20260928 14:37:56.547874 24133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21e5aada8e6a47bc8d49c18052d17258" candidate_uuid: "6bf35196da0740e7b5900e1c7f842a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa3b206cad4423087e0646de8008b85"
I20260928 14:37:56.547951 23853 raft_consensus.cc:3037] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:56.547960 24133 raft_consensus.cc:3037] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:56.548561 23853 raft_consensus.cc:2454] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bf35196da0740e7b5900e1c7f842a53 in term 1.
I20260928 14:37:56.548735 23926 leader_election.cc:302] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bf35196da0740e7b5900e1c7f842a53, c12c9d707cf24a189e31a7b2defa6d9f; no voters:
I20260928 14:37:56.548841 24207 raft_consensus.cc:2781] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:37:56.548863 24133 raft_consensus.cc:2454] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bf35196da0740e7b5900e1c7f842a53 in term 1.
I20260928 14:37:56.548977 24207 raft_consensus.cc:686] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [term 1 LEADER]: Becoming Leader. State: Replica: 6bf35196da0740e7b5900e1c7f842a53, State: Running, Role: LEADER
I20260928 14:37:56.549055 24207 consensus_queue.cc:237] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.549778 23696 catalog_manager.cc:5495] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bf35196da0740e7b5900e1c7f842a53 (127.22.13.66). New cstate: current_term: 1 leader_uuid: "6bf35196da0740e7b5900e1c7f842a53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } health_report { overall_health: UNKNOWN } } }
Sep 28 14:37:56 dist-test-slave-wwwb krb5kdc[23651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606275, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:56.589720 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.582793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33192 (local address 127.22.13.126:41281)
0928 14:37:56.582858 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.582860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.583013 (+ 153us) server_negotiation.cc:407] Connection header received
0928 14:37:56.583089 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.583090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.583100 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.583152 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:56.583855 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.584433 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.585076 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.585280 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.585340 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.587397 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:56.587398 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:56.587398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:56.587405 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:56.588770 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:56.589115 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:56.589116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:56.589116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:56.589164 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:56.589322 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:56.589323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:56.589323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:56.589423 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:56.589470 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.589585 (+ 115us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.589653 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260928 14:37:56.589854 24240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.582727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41281 (local address 127.0.0.1:33192)
0928 14:37:56.582915 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:56.582927 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:56.583025 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:56.583186 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:56.583187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:56.583646 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:56.583716 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.583719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.584495 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.584496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.585003 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.585005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.585351 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.585352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.585394 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.585884 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:56.585889 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:56.587304 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:56.588843 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:56.588850 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:56.588852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:56.589054 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:56.589216 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:56.589217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:56.589217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:56.589270 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:56.589472 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:56.589478 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:56.589521 (+ 43us) client_negotiation.cc:769] Sending connection context
0928 14:37:56.589571 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 14:37:56.589703 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":44,"thread_start_us":89,"threads_started":1}
I20260928 14:37:56.591718 23696 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:33192:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 14:37:56.591903 23696 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:37:56.593386 23973 tablet_service.cc:1467] Processing CreateTablet for tablet 828d96439ecf4066bd5a1ed460e78fd2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e38f8010a15e42ef8fedb817c3dd24b8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:37:56.593398 24113 tablet_service.cc:1467] Processing CreateTablet for tablet 828d96439ecf4066bd5a1ed460e78fd2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e38f8010a15e42ef8fedb817c3dd24b8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:37:56.593397 23833 tablet_service.cc:1467] Processing CreateTablet for tablet 828d96439ecf4066bd5a1ed460e78fd2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e38f8010a15e42ef8fedb817c3dd24b8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:37:56.593562 24113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828d96439ecf4066bd5a1ed460e78fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:56.593567 23973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828d96439ecf4066bd5a1ed460e78fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:56.593580 23833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828d96439ecf4066bd5a1ed460e78fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:56.594913 24203 tablet_bootstrap.cc:492] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f: Bootstrap starting.
I20260928 14:37:56.595104 24202 tablet_bootstrap.cc:492] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53: Bootstrap starting.
I20260928 14:37:56.595104 24206 tablet_bootstrap.cc:492] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85: Bootstrap starting.
I20260928 14:37:56.595391 24203 tablet_bootstrap.cc:654] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:56.595466 24206 tablet_bootstrap.cc:654] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:56.595484 24202 tablet_bootstrap.cc:654] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:56.596035 24206 tablet_bootstrap.cc:492] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85: No bootstrap required, opened a new log
I20260928 14:37:56.596089 24203 tablet_bootstrap.cc:492] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f: No bootstrap required, opened a new log
I20260928 14:37:56.596095 24206 ts_tablet_manager.cc:1397] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:37:56.596130 24203 ts_tablet_manager.cc:1397] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:37:56.596135 24202 tablet_bootstrap.cc:492] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53: No bootstrap required, opened a new log
I20260928 14:37:56.596180 24202 ts_tablet_manager.cc:1397] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:37:56.596222 24206 raft_consensus.cc:348] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.596316 24206 raft_consensus.cc:374] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:56.596273 24203 raft_consensus.cc:348] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.596339 24206 raft_consensus.cc:729] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa3b206cad4423087e0646de8008b85, State: Initialized, Role: FOLLOWER
I20260928 14:37:56.596341 24203 raft_consensus.cc:374] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:56.596365 24203 raft_consensus.cc:729] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c12c9d707cf24a189e31a7b2defa6d9f, State: Initialized, Role: FOLLOWER
I20260928 14:37:56.596357 24202 raft_consensus.cc:348] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.596391 24206 consensus_queue.cc:260] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.596410 24203 consensus_queue.cc:260] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.596442 24202 raft_consensus.cc:374] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:56.596477 24202 raft_consensus.cc:729] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bf35196da0740e7b5900e1c7f842a53, State: Initialized, Role: FOLLOWER
I20260928 14:37:56.596484 24206 ts_tablet_manager.cc:1428] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:56.596496 24203 ts_tablet_manager.cc:1428] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:56.596521 24202 consensus_queue.cc:260] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.596526 24220 tablet_replica.cc:439] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } }
I20260928 14:37:56.596616 24202 ts_tablet_manager.cc:1428] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:56.596612 24220 tablet_replica.cc:442] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:37:56.596668 24207 tablet_replica.cc:439] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } }
I20260928 14:37:56.596757 24207 tablet_replica.cc:442] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:37:56.596938 24209 tablet_replica.cc:439] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } }
I20260928 14:37:56.597031 24209 tablet_replica.cc:442] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:37:56.784307 24220 raft_consensus.cc:482] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:37:56.784412 24220 raft_consensus.cc:504] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.784628 24220 leader_election.cc:288] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c12c9d707cf24a189e31a7b2defa6d9f (127.22.13.65:40759), 6bf35196da0740e7b5900e1c7f842a53 (127.22.13.66:39683)
I20260928 14:37:56.784874 23853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828d96439ecf4066bd5a1ed460e78fd2" candidate_uuid: "5fa3b206cad4423087e0646de8008b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" is_pre_election: true
I20260928 14:37:56.784874 23993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828d96439ecf4066bd5a1ed460e78fd2" candidate_uuid: "5fa3b206cad4423087e0646de8008b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bf35196da0740e7b5900e1c7f842a53" is_pre_election: true
I20260928 14:37:56.784979 23993 raft_consensus.cc:2454] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fa3b206cad4423087e0646de8008b85 in term 0.
I20260928 14:37:56.784979 23853 raft_consensus.cc:2454] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fa3b206cad4423087e0646de8008b85 in term 0.
I20260928 14:37:56.785153 24067 leader_election.cc:302] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa3b206cad4423087e0646de8008b85, 6bf35196da0740e7b5900e1c7f842a53; no voters:
I20260928 14:37:56.785274 24220 raft_consensus.cc:2781] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:37:56.785318 24220 raft_consensus.cc:482] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:37:56.785337 24220 raft_consensus.cc:3037] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:56.786243 24220 raft_consensus.cc:504] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.786396 24220 leader_election.cc:288] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [CANDIDATE]: Term 1 election: Requested vote from peers c12c9d707cf24a189e31a7b2defa6d9f (127.22.13.65:40759), 6bf35196da0740e7b5900e1c7f842a53 (127.22.13.66:39683)
I20260928 14:37:56.786605 23993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828d96439ecf4066bd5a1ed460e78fd2" candidate_uuid: "5fa3b206cad4423087e0646de8008b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bf35196da0740e7b5900e1c7f842a53"
I20260928 14:37:56.786612 23853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828d96439ecf4066bd5a1ed460e78fd2" candidate_uuid: "5fa3b206cad4423087e0646de8008b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12c9d707cf24a189e31a7b2defa6d9f"
I20260928 14:37:56.786686 23993 raft_consensus.cc:3037] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:56.786703 23853 raft_consensus.cc:3037] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:56.787313 23993 raft_consensus.cc:2454] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fa3b206cad4423087e0646de8008b85 in term 1.
I20260928 14:37:56.787477 23853 raft_consensus.cc:2454] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fa3b206cad4423087e0646de8008b85 in term 1.
I20260928 14:37:56.787514 24067 leader_election.cc:302] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa3b206cad4423087e0646de8008b85, 6bf35196da0740e7b5900e1c7f842a53; no voters:
I20260928 14:37:56.787602 24220 raft_consensus.cc:2781] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:37:56.787644 24220 raft_consensus.cc:686] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [term 1 LEADER]: Becoming Leader. State: Replica: 5fa3b206cad4423087e0646de8008b85, State: Running, Role: LEADER
I20260928 14:37:56.787699 24220 consensus_queue.cc:237] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } }
I20260928 14:37:56.787985 24220 tablet_replica.cc:439] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85: TxnStatusTablet state changed. Reason: New leader 5fa3b206cad4423087e0646de8008b85. Latest consensus state: current_term: 1 leader_uuid: "5fa3b206cad4423087e0646de8008b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } }
I20260928 14:37:56.788060 24220 tablet_replica.cc:442] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:37:56.788203 24245 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 14:37:56.788309 24245 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 14:37:56.788345 23694 catalog_manager.cc:5495] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fa3b206cad4423087e0646de8008b85 (127.22.13.67). New cstate: current_term: 1 leader_uuid: "5fa3b206cad4423087e0646de8008b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } health_report { overall_health: HEALTHY } } }
I20260928 14:37:56.825663 24240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.822939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:46677 (local address 127.0.0.1:50272)
0928 14:37:56.823035 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:56.823048 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:56.823106 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:56.823247 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:56.823248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:56.823249 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:37:56.823304 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.823306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.824194 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.824194 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.824784 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.824786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.825267 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.825268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.825313 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.825333 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:37:56.825519 (+ 186us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:37:56.825526 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 14:37:56.825557 (+ 31us) client_negotiation.cc:240] Negotiation successful
0928 14:37:56.825600 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260928 14:37:56.825676 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.823000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50272 (local address 127.22.13.67:46677)
0928 14:37:56.823054 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.823056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.823093 (+ 37us) server_negotiation.cc:407] Connection header received
0928 14:37:56.823164 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.823165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.823176 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.823235 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:37:56.823364 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.824112 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.824874 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.825182 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.825249 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.825423 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:37:56.825460 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:37:56.825514 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.825573 (+ 59us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.825593 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 14:37:56.826709 23853 raft_consensus.cc:1261] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f [term 1 FOLLOWER]: Refusing update from remote peer 5fa3b206cad4423087e0646de8008b85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:37:56.826733 23993 raft_consensus.cc:1261] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53 [term 1 FOLLOWER]: Refusing update from remote peer 5fa3b206cad4423087e0646de8008b85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:37:56.826977 24220 consensus_queue.cc:1035] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:37:56.827131 24210 consensus_queue.cc:1035] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:37:56.827744 24209 tablet_replica.cc:439] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f: TxnStatusTablet state changed. Reason: New leader 5fa3b206cad4423087e0646de8008b85. Latest consensus state: current_term: 1 leader_uuid: "5fa3b206cad4423087e0646de8008b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } }
I20260928 14:37:56.827862 24209 tablet_replica.cc:442] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:37:56.828135 24207 tablet_replica.cc:439] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53: TxnStatusTablet state changed. Reason: New leader 5fa3b206cad4423087e0646de8008b85. Latest consensus state: current_term: 1 leader_uuid: "5fa3b206cad4423087e0646de8008b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } }
I20260928 14:37:56.828239 24207 tablet_replica.cc:442] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:37:56.828728 24220 tablet_replica.cc:439] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5fa3b206cad4423087e0646de8008b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } }
I20260928 14:37:56.828815 24220 tablet_replica.cc:442] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:37:56.828886 24220 tablet_replica.cc:439] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5fa3b206cad4423087e0646de8008b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } }
I20260928 14:37:56.828929 24220 tablet_replica.cc:442] T 828d96439ecf4066bd5a1ed460e78fd2 P 5fa3b206cad4423087e0646de8008b85: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:37:56.829257 24209 tablet_replica.cc:439] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5fa3b206cad4423087e0646de8008b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } }
I20260928 14:37:56.829331 24209 tablet_replica.cc:442] T 828d96439ecf4066bd5a1ed460e78fd2 P c12c9d707cf24a189e31a7b2defa6d9f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:37:56.829336 24207 tablet_replica.cc:439] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5fa3b206cad4423087e0646de8008b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 } } peers { permanent_uuid: "6bf35196da0740e7b5900e1c7f842a53" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39683 } } peers { permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 } } }
I20260928 14:37:56.829432 24207 tablet_replica.cc:442] T 828d96439ecf4066bd5a1ed460e78fd2 P 6bf35196da0740e7b5900e1c7f842a53: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:37:56.833411 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.830871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37596 (local address 127.22.13.66:39683)
0928 14:37:56.830907 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.830909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.830918 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:37:56.830992 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.830993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.831009 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.831070 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:37:56.831175 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.831639 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.832719 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.832899 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.832948 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.833112 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:37:56.833216 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:37:56.833271 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.833338 (+ 67us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.833355 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 14:37:56.838550 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.836044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:35617 (local address 127.22.13.67:46677)
0928 14:37:56.836104 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.836106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.836216 (+ 110us) server_negotiation.cc:407] Connection header received
0928 14:37:56.836296 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.836297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.836308 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.836364 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:37:56.836515 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.836992 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.837762 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.838040 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.838124 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.838304 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:37:56.838338 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:37:56.838387 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.838450 (+ 63us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.838470 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:37:56.838581 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.835986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:46677 (local address 127.22.13.66:35617)
0928 14:37:56.836144 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:56.836155 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:56.836233 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:56.836366 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:56.836367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:56.836369 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:37:56.836437 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.836438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.837072 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.837073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.837673 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.837674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.838125 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.838126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.838194 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.838214 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:37:56.838392 (+ 178us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:37:56.838394 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:37:56.838445 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 14:37:56.838491 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
I20260928 14:37:56.841090 23853 raft_consensus.cc:1261] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f [term 1 FOLLOWER]: Refusing update from remote peer 6bf35196da0740e7b5900e1c7f842a53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:37:56.841195 24133 raft_consensus.cc:1261] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85 [term 1 FOLLOWER]: Refusing update from remote peer 6bf35196da0740e7b5900e1c7f842a53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:37:56.841514 24233 consensus_queue.cc:1035] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c12c9d707cf24a189e31a7b2defa6d9f" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:37:56.841668 24207 consensus_queue.cc:1035] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fa3b206cad4423087e0646de8008b85" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 46677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:37:56.844007 24216 mvcc.cc:205] Tried to move back new op lower bound from 7334323309939204096 to 7334323308745678848. Current Snapshot: MvccSnapshot[applied={T|T < 7334323309939204096}]
I20260928 14:37:56.850528 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.847963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:40497 (local address 127.22.13.66:39683)
0928 14:37:56.848005 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.848007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.848121 (+ 114us) server_negotiation.cc:407] Connection header received
0928 14:37:56.848190 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.848190 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.848200 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.848239 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:37:56.848367 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.848825 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.849827 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.850088 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.850143 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.850321 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:37:56.850349 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:37:56.850382 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.850448 (+ 66us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.850468 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:37:56.850575 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.847898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:39683 (local address 127.22.13.67:40497)
0928 14:37:56.848058 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:56.848070 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:56.848133 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:56.848257 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:56.848258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:56.848259 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:37:56.848311 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.848313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.848904 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.848905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.849714 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:56.849718 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.850159 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.850160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:56.850217 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.850235 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:37:56.850386 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:37:56.850388 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:37:56.850446 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 14:37:56.850491 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260928 14:37:56.861074 23695 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:37:56.861210 23695 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:37:56.861928 23695 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0563d78eac4f447686579df1b281f9aa: Sending DeleteTablet for 3 replicas of tablet 21e5aada8e6a47bc8d49c18052d17258
I20260928 14:37:56.862246 23973 tablet_service.cc:1514] Processing DeleteTablet for tablet 21e5aada8e6a47bc8d49c18052d17258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:37:56 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:37592
I20260928 14:37:56.862341 24227 tablet_replica.cc:330] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53: stopping tablet replica
I20260928 14:37:56.862392 24227 raft_consensus.cc:2229] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:37:56.862490 24227 raft_consensus.cc:2258] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:37:56.862587 23833 tablet_service.cc:1514] Processing DeleteTablet for tablet 21e5aada8e6a47bc8d49c18052d17258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:37:56 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:50044
I20260928 14:37:56.862692 24113 tablet_service.cc:1514] Processing DeleteTablet for tablet 21e5aada8e6a47bc8d49c18052d17258 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:37:56 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:50254
I20260928 14:37:56.862689 24225 tablet_replica.cc:330] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f: stopping tablet replica
I20260928 14:37:56.862776 24225 raft_consensus.cc:2229] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:37:56.862789 24226 tablet_replica.cc:330] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85: stopping tablet replica
I20260928 14:37:56.862821 24225 raft_consensus.cc:2258] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:37:56.862852 24226 raft_consensus.cc:2229] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:37:56.862895 24226 raft_consensus.cc:2258] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:37:56.863250 24227 ts_tablet_manager.cc:1905] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:37:56.863279 24225 ts_tablet_manager.cc:1905] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:37:56.863339 24226 ts_tablet_manager.cc:1905] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:37:56.865059 24226 ts_tablet_manager.cc:1918] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:37:56.865060 24225 ts_tablet_manager.cc:1918] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:37:56.865124 24226 log.cc:1196] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/21e5aada8e6a47bc8d49c18052d17258
I20260928 14:37:56.865126 24225 log.cc:1196] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-0/wal/wals/21e5aada8e6a47bc8d49c18052d17258
I20260928 14:37:56.865356 24225 ts_tablet_manager.cc:1939] T 21e5aada8e6a47bc8d49c18052d17258 P c12c9d707cf24a189e31a7b2defa6d9f: Deleting consensus metadata
I20260928 14:37:56.865374 24226 ts_tablet_manager.cc:1939] T 21e5aada8e6a47bc8d49c18052d17258 P 5fa3b206cad4423087e0646de8008b85: Deleting consensus metadata
I20260928 14:37:56.865718 23682 catalog_manager.cc:4841] TS c12c9d707cf24a189e31a7b2defa6d9f (127.22.13.65:40759): tablet 21e5aada8e6a47bc8d49c18052d17258 (table test-table [id=8c458cf1f008464db91e6137f5ba9e25]) successfully deleted
I20260928 14:37:56.865803 24227 ts_tablet_manager.cc:1918] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:37:56.865870 24227 log.cc:1196] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/21e5aada8e6a47bc8d49c18052d17258
I20260928 14:37:56.866127 24227 ts_tablet_manager.cc:1939] T 21e5aada8e6a47bc8d49c18052d17258 P 6bf35196da0740e7b5900e1c7f842a53: Deleting consensus metadata
I20260928 14:37:56.865720 23680 catalog_manager.cc:4841] TS 5fa3b206cad4423087e0646de8008b85 (127.22.13.67:46677): tablet 21e5aada8e6a47bc8d49c18052d17258 (table test-table [id=8c458cf1f008464db91e6137f5ba9e25]) successfully deleted
I20260928 14:37:56.866485 23683 catalog_manager.cc:4841] TS 6bf35196da0740e7b5900e1c7f842a53 (127.22.13.66:39683): tablet 21e5aada8e6a47bc8d49c18052d17258 (table test-table [id=8c458cf1f008464db91e6137f5ba9e25]) successfully deleted
Sep 28 14:37:56 dist-test-slave-wwwb krb5kdc[23651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606276, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.13.65@KRBTEST.COM
I20260928 14:37:56.880916 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.866011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50056 (local address 127.22.13.65:40759)
0928 14:37:56.866384 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.866386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.870190 (+ 3804us) server_negotiation.cc:407] Connection header received
0928 14:37:56.870279 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.870281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.870299 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.870361 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:56.871147 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.871899 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.872997 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.873312 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.873389 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.877107 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:56.877123 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:56.877124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:56.877134 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:56.879328 (+ 2194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:56.879782 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:56.879787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:56.879787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:56.879876 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:56.880121 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:56.880122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:56.880123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:56.880327 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:56.880472 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.880715 (+ 243us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.880813 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349}
W20260928 14:37:56.881821 23876 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50056
I20260928 14:37:56.894444 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:56.886255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33204 (local address 127.22.13.126:41281)
0928 14:37:56.886305 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:56.886307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:56.886442 (+ 135us) server_negotiation.cc:407] Connection header received
0928 14:37:56.886556 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:56.886557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:56.886569 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:56.886601 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:56.887441 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.888221 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.889364 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:56.889674 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:56.889724 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:56.891385 (+ 1661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:56.891387 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:56.891387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:56.891396 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:56.893085 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:56.893504 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:56.893505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:56.893506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:56.893564 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:56.893757 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:56.893758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:56.893758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:56.893895 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:56.894019 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:56.894249 (+ 230us) server_negotiation.cc:299] Negotiation successful
0928 14:37:56.894332 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260928 14:37:56.895038 23695 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33204
I20260928 14:37:56.895736 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 23768
I20260928 14:37:56.903184 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 23908
I20260928 14:37:56.907979 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 24048
I20260928 14:37:56.912870 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 23664
2026-09-28T14:37:56Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2646 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:37:56 dist-test-slave-wwwb krb5kdc[24266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:37:56 dist-test-slave-wwwb krb5kdc[24266](info): set up 2 sockets
Sep 28 14:37:56 dist-test-slave-wwwb krb5kdc[24266](info): commencing operation
krb5kdc: starting...
W20260928 14:37:57.934090 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:37:58 dist-test-slave-wwwb krb5kdc[24266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:37:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:37:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:37:58.056432 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:37895
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:37895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:37:58.139011 24279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:37:58.139287 24279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:37:58.139334 24279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:37:58.140503 24279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:37:58.140583 24279 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 14:37:58.140604 24279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:37:58.140626 24279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:37:58.140651 24279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:37:58.140681 24279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:37:58.142275 24279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:37895
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:37895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:37:58.142591 24279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:37:58.142832 24279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:37:58.147882 24287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:58.147852 24285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:58.147886 24284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:58.148082 24279 server_base.cc:1034] running on GCE node
I20260928 14:37:58.148458 24279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:37:58.148731 24279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:37:58.149905 24279 hybrid_clock.cc:648] HybridClock initialized: now 1790606278149901 us; error 39 us; skew 500 ppm
Sep 28 14:37:58 dist-test-slave-wwwb krb5kdc[24266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606278, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:37:58.152460 24279 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:37:58.152901 24279 webserver.cc:460] Webserver started at http://127.22.13.126:45561/ using document root <none> and password file <none>
I20260928 14:37:58.153152 24279 fs_manager.cc:359] Metadata directory not provided
I20260928 14:37:58.153221 24279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:37:58.153368 24279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:37:58.154531 24279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "2a3dd7775ab7468d807ddd3c8828a3a7"
format_stamp: "Formatted at 2026-09-28 14:37:58 on dist-test-slave-wwwb"
server_key: "6adcf4a2cd4bf82d759a8cbd7de7adb2"
server_key_iv: "218be3d5e9cda09b9acb1883c0dc7ab6"
server_key_version: "encryptionkey@0"
I20260928 14:37:58.154896 24279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "2a3dd7775ab7468d807ddd3c8828a3a7"
format_stamp: "Formatted at 2026-09-28 14:37:58 on dist-test-slave-wwwb"
server_key: "6adcf4a2cd4bf82d759a8cbd7de7adb2"
server_key_iv: "218be3d5e9cda09b9acb1883c0dc7ab6"
server_key_version: "encryptionkey@0"
I20260928 14:37:58.156250 24279 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:37:58.156975 24294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:58.157150 24279 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:37:58.157254 24279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "2a3dd7775ab7468d807ddd3c8828a3a7"
format_stamp: "Formatted at 2026-09-28 14:37:58 on dist-test-slave-wwwb"
server_key: "6adcf4a2cd4bf82d759a8cbd7de7adb2"
server_key_iv: "218be3d5e9cda09b9acb1883c0dc7ab6"
server_key_version: "encryptionkey@0"
I20260928 14:37:58.157331 24279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:37:58.170529 24279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:37:58.172474 24279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:37:58.172643 24279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:37:58.177277 24279 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:37895
I20260928 14:37:58.177356 24356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:37895 every 8 connection(s)
I20260928 14:37:58.177693 24279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:37:58.178454 24357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:58.180732 24357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7: Bootstrap starting.
I20260928 14:37:58.181367 24357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:58.181540 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 24279
I20260928 14:37:58.181639 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:37:58.181711 24357 log.cc:824] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7: Log is configured to *not* fsync() on all Append() calls
I20260928 14:37:58.181830 22581 external_mini_cluster.cc:1428] Setting key 40f6de88e761d2075fb0a69757cd8798
I20260928 14:37:58.182559 24357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7: No bootstrap required, opened a new log
I20260928 14:37:58.184755 24357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3dd7775ab7468d807ddd3c8828a3a7" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37895 } }
I20260928 14:37:58.184962 24357 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:58.185016 24357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a3dd7775ab7468d807ddd3c8828a3a7, State: Initialized, Role: FOLLOWER
I20260928 14:37:58.185153 24357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3dd7775ab7468d807ddd3c8828a3a7" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37895 } }
I20260928 14:37:58.185235 24357 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:37:58.185269 24357 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:37:58.185344 24357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:58.185884 24357 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3dd7775ab7468d807ddd3c8828a3a7" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37895 } }
I20260928 14:37:58.186033 24357 leader_election.cc:302] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a3dd7775ab7468d807ddd3c8828a3a7; no voters:
I20260928 14:37:58.186226 24357 leader_election.cc:288] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:37:58.186251 24362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:37:58.186362 24362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [term 1 LEADER]: Becoming Leader. State: Replica: 2a3dd7775ab7468d807ddd3c8828a3a7, State: Running, Role: LEADER
I20260928 14:37:58.186453 24362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3dd7775ab7468d807ddd3c8828a3a7" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37895 } }
I20260928 14:37:58.186512 24357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:37:58.186748 24362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a3dd7775ab7468d807ddd3c8828a3a7. Latest consensus state: current_term: 1 leader_uuid: "2a3dd7775ab7468d807ddd3c8828a3a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3dd7775ab7468d807ddd3c8828a3a7" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37895 } } }
I20260928 14:37:58.186900 24362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [sys.catalog]: This master's current role is: LEADER
I20260928 14:37:58.186760 24364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a3dd7775ab7468d807ddd3c8828a3a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a3dd7775ab7468d807ddd3c8828a3a7" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37895 } } }
I20260928 14:37:58.187124 24364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7 [sys.catalog]: This master's current role is: LEADER
I20260928 14:37:58.187243 24367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:37:58.187747 24367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 28 14:37:58 dist-test-slave-wwwb krb5kdc[24266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:58.189442 24367 catalog_manager.cc:1347] Generated new cluster ID: 3ea537c4da1549b381084496d94f5301
I20260928 14:37:58.189505 24367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:37:58.193145 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.182365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60602 (local address 127.22.13.126:37895)
0928 14:37:58.182557 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:58.182567 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:58.182608 (+ 41us) server_negotiation.cc:407] Connection header received
0928 14:37:58.183110 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:58.183117 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:58.183167 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:58.183318 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:58.183960 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.184837 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.185758 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.186338 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.186452 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.189527 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:58.189531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:58.189531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:58.189543 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:58.191460 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.191899 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.191901 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.191902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.192009 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.192226 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.192226 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.192227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.192521 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:58.192675 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:58.192801 (+ 126us) server_negotiation.cc:299] Negotiation successful
0928 14:37:58.192933 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260928 14:37:58.201421 24367 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:37:58.202050 24367 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:37:58.204181 24367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7: Generated new TSK 0
I20260928 14:37:58.204373 24367 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:37:58.230300 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:37895
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:37:58.313941 24383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:37:58.314189 24383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:37:58.314216 24383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:37:58.315506 24383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:37:58.315553 24383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:37:58.315575 24383 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 14:37:58.315587 24383 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:37:58.315605 24383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:37:58.317021 24383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.13.126:37895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:37:58.317217 24383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:37:58.317420 24383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:37:58.322407 24389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:58.322431 24391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:58.322512 24388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:58.322880 24383 server_base.cc:1034] running on GCE node
I20260928 14:37:58.323083 24383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:37:58.323345 24383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:37:58.324517 24383 hybrid_clock.cc:648] HybridClock initialized: now 1790606278324511 us; error 48 us; skew 500 ppm
Sep 28 14:37:58 dist-test-slave-wwwb krb5kdc[24266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606278, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:37:58.327073 24383 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:37:58.327540 24383 webserver.cc:460] Webserver started at http://127.22.13.65:37725/ using document root <none> and password file <none>
I20260928 14:37:58.327777 24383 fs_manager.cc:359] Metadata directory not provided
I20260928 14:37:58.327848 24383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:37:58.328006 24383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:37:58.329093 24383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "86bd41d8b4634545a7c4e96e5e50f2d0"
format_stamp: "Formatted at 2026-09-28 14:37:58 on dist-test-slave-wwwb"
server_key: "37c36e88ec56b839f2d24fdcd5f3238c"
server_key_iv: "83de6aff8f369da194a18bf390e445f1"
server_key_version: "encryptionkey@0"
I20260928 14:37:58.329478 24383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "86bd41d8b4634545a7c4e96e5e50f2d0"
format_stamp: "Formatted at 2026-09-28 14:37:58 on dist-test-slave-wwwb"
server_key: "37c36e88ec56b839f2d24fdcd5f3238c"
server_key_iv: "83de6aff8f369da194a18bf390e445f1"
server_key_version: "encryptionkey@0"
I20260928 14:37:58.330907 24383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:37:58.331629 24398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:58.331799 24383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:37:58.331924 24383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "86bd41d8b4634545a7c4e96e5e50f2d0"
format_stamp: "Formatted at 2026-09-28 14:37:58 on dist-test-slave-wwwb"
server_key: "37c36e88ec56b839f2d24fdcd5f3238c"
server_key_iv: "83de6aff8f369da194a18bf390e445f1"
server_key_version: "encryptionkey@0"
I20260928 14:37:58.332005 24383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:37:58.341915 24383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:37:58.343840 24383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:37:58.344029 24383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:37:58.344590 24383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:37:58.344663 24383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:58.344696 24383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:37:58.344750 24383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:58.351716 24383 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:38091
I20260928 14:37:58.351790 24513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:38091 every 8 connection(s)
I20260928 14:37:58.352150 24383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
Sep 28 14:37:58 dist-test-slave-wwwb krb5kdc[24266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606278, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:58.355398 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.345093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:42981 (local address 127.22.13.126:37895)
0928 14:37:58.345167 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:58.345170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:58.345447 (+ 277us) server_negotiation.cc:407] Connection header received
0928 14:37:58.346037 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:58.346038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:58.346051 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:58.346109 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:58.347061 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.347916 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.349623 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.350002 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.350099 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.352767 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:58.352769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:58.352769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:58.352777 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:58.354070 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.354461 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.354461 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.354462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.354516 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.354731 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.354731 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.354732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.354840 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:58.354890 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:58.355259 (+ 369us) server_negotiation.cc:299] Negotiation successful
0928 14:37:58.355331 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:37:58.355410 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 24383
I20260928 14:37:58.355501 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:37:58.355681 22581 external_mini_cluster.cc:1428] Setting key 1de944a2c67c9213d8f865f6ffd909a6
I20260928 14:37:58.355713 24407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.345079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:37895 (local address 127.22.13.65:42981)
0928 14:37:58.345333 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:58.345365 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:58.345883 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:58.346191 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:58.346195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:58.346601 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:58.346927 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.346933 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.348063 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.348063 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.349510 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.349514 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.350101 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.350102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.350220 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.350796 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:58.350808 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:58.352667 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:58.354151 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:58.354157 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:58.354159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:58.354390 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:58.354574 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:58.354574 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:58.354575 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:58.354677 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:58.354894 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:58.354898 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:58.355025 (+ 127us) client_negotiation.cc:769] Sending connection context
0928 14:37:58.355249 (+ 224us) client_negotiation.cc:240] Negotiation successful
0928 14:37:58.355464 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":89,"threads_started":1}
I20260928 14:37:58.356261 24514 heartbeater.cc:347] Connected to a master server at 127.22.13.126:37895
I20260928 14:37:58.356359 24514 heartbeater.cc:464] Registering TS with master...
I20260928 14:37:58.356559 24514 heartbeater.cc:511] Master 127.22.13.126:37895 requested a full tablet report, sending...
I20260928 14:37:58.357123 24311 ts_manager.cc:194] Registered new tserver with Master: 86bd41d8b4634545a7c4e96e5e50f2d0 (127.22.13.65:38091)
I20260928 14:37:58.358086 24311 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:42981
I20260928 14:37:58.364641 24520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.357329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:37895 (local address 127.22.13.65:52975)
0928 14:37:58.357458 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:58.357469 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:58.357522 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:58.359008 (+ 1486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:58.359009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:58.359246 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:58.359319 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.359321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.360023 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.360024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.360631 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.360632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.361032 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.361033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.361115 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.361585 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:58.361589 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:58.361888 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:58.363540 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:58.363541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:58.363542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:58.363730 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:58.363950 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:58.363951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:58.363951 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:58.364016 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:58.364269 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:58.364271 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:58.364320 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 14:37:58.364418 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 14:37:58.364538 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
I20260928 14:37:58.364635 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.358792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:52975 (local address 127.22.13.126:37895)
0928 14:37:58.358857 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:58.358859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:58.358869 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:37:58.358892 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:58.358892 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:58.358903 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:58.358977 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:58.359419 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.359925 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.360711 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.360971 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.361035 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.362073 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:58.362074 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:58.362075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:58.362082 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:58.363430 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.363806 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.363807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.363808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.363875 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.364078 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.364079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.364079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.364195 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:58.364267 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:58.364451 (+ 184us) server_negotiation.cc:299] Negotiation successful
0928 14:37:58.364535 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:37:58.382841 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:37895
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:37:58.466446 24523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:37:58.466698 24523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:37:58.466739 24523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:37:58.467978 24523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:37:58.468052 24523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:37:58.468087 24523 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 14:37:58.468113 24523 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:37:58.468168 24523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:37:58.469624 24523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.13.126:37895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:37:58.469903 24523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:37:58.470163 24523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:37:58.475392 24529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:58.475399 24531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:58.475400 24528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:58.475674 24523 server_base.cc:1034] running on GCE node
I20260928 14:37:58.475893 24523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:37:58.476190 24523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:37:58.477358 24523 hybrid_clock.cc:648] HybridClock initialized: now 1790606278477354 us; error 37 us; skew 500 ppm
Sep 28 14:37:58 dist-test-slave-wwwb krb5kdc[24266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606278, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:37:58.479957 24523 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:37:58.480456 24523 webserver.cc:460] Webserver started at http://127.22.13.66:46025/ using document root <none> and password file <none>
I20260928 14:37:58.480707 24523 fs_manager.cc:359] Metadata directory not provided
I20260928 14:37:58.480777 24523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:37:58.480926 24523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:37:58.482067 24523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "abadc8072e634fd98393c935e8ed6d21"
format_stamp: "Formatted at 2026-09-28 14:37:58 on dist-test-slave-wwwb"
server_key: "9a014a155cfe2db0fbc621ac9e7d3367"
server_key_iv: "9ad941874c08f8c44457ad46c00aafac"
server_key_version: "encryptionkey@0"
I20260928 14:37:58.482434 24523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "abadc8072e634fd98393c935e8ed6d21"
format_stamp: "Formatted at 2026-09-28 14:37:58 on dist-test-slave-wwwb"
server_key: "9a014a155cfe2db0fbc621ac9e7d3367"
server_key_iv: "9ad941874c08f8c44457ad46c00aafac"
server_key_version: "encryptionkey@0"
I20260928 14:37:58.483829 24523 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:37:58.484632 24538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:58.484812 24523 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:37:58.484906 24523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "abadc8072e634fd98393c935e8ed6d21"
format_stamp: "Formatted at 2026-09-28 14:37:58 on dist-test-slave-wwwb"
server_key: "9a014a155cfe2db0fbc621ac9e7d3367"
server_key_iv: "9ad941874c08f8c44457ad46c00aafac"
server_key_version: "encryptionkey@0"
I20260928 14:37:58.484993 24523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:37:58.506264 24523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:37:58.508277 24523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:37:58.508451 24523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:37:58.509032 24523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:37:58.509091 24523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:58.509133 24523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:37:58.509166 24523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:37:58 dist-test-slave-wwwb krb5kdc[24266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606278, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:58.516510 24523 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:38305
I20260928 14:37:58.516546 24653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:38305 every 8 connection(s)
I20260928 14:37:58.516872 24523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:37:58.518096 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 24523
I20260928 14:37:58.518209 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:37:58.518388 22581 external_mini_cluster.cc:1428] Setting key b02b603f76d4079ad1ec0b86b457194d
I20260928 14:37:58.519204 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.509468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:33015 (local address 127.22.13.126:37895)
0928 14:37:58.509513 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:58.509514 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:58.509751 (+ 237us) server_negotiation.cc:407] Connection header received
0928 14:37:58.510374 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:58.510376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:58.510390 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:58.510443 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:58.511153 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.511653 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.512695 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.512924 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.513013 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.516012 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:58.516013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:58.516014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:58.516022 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:58.517586 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.518157 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.518158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.518159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.518224 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.518486 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.518487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.518488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.518618 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:58.518693 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:58.519030 (+ 337us) server_negotiation.cc:299] Negotiation successful
0928 14:37:58.519115 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:37:58.519409 24547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.509473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:37895 (local address 127.22.13.66:33015)
0928 14:37:58.509629 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:58.509665 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:58.510253 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:58.510544 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:58.510548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:58.510857 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:58.511056 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.511060 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.511752 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.511753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.512597 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.512599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.513024 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.513025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.513189 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.513739 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:58.513747 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:58.515902 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:58.517687 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:58.517693 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:58.517695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:58.518078 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:58.518293 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:58.518293 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:58.518293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:58.518412 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:58.518694 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:58.518701 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:58.518885 (+ 184us) client_negotiation.cc:769] Sending connection context
0928 14:37:58.518994 (+ 109us) client_negotiation.cc:240] Negotiation successful
0928 14:37:58.519162 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260928 14:37:58.520687 24654 heartbeater.cc:347] Connected to a master server at 127.22.13.126:37895
I20260928 14:37:58.520780 24654 heartbeater.cc:464] Registering TS with master...
I20260928 14:37:58.520954 24654 heartbeater.cc:511] Master 127.22.13.126:37895 requested a full tablet report, sending...
I20260928 14:37:58.521340 24309 ts_manager.cc:194] Registered new tserver with Master: abadc8072e634fd98393c935e8ed6d21 (127.22.13.66:38305)
I20260928 14:37:58.521857 24309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:33015
I20260928 14:37:58.527381 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.521236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:50001 (local address 127.22.13.126:37895)
0928 14:37:58.521277 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:58.521280 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:58.521287 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:37:58.521309 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:58.521310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:58.521320 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:58.521380 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:58.521786 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.522518 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.523355 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.523580 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.523665 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.524717 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:58.524719 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:58.524719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:58.524726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:58.526171 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.526580 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.526581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.526581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.526634 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.526858 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.526858 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.526858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.527013 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:58.527064 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:58.527256 (+ 192us) server_negotiation.cc:299] Negotiation successful
0928 14:37:58.527316 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:37:58.527522 24661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.521035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:37895 (local address 127.22.13.66:50001)
0928 14:37:58.521178 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:58.521189 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:58.521238 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:58.521394 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:58.521395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:58.521638 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:58.521697 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.521698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.522631 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.522632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.523266 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.523269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.523667 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.523668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.523748 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.524157 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:58.524162 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:58.524615 (+ 453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:58.526267 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:58.526268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:58.526268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:58.526508 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:58.526698 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:58.526699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:58.526699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:58.526792 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:58.527082 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:58.527084 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:58.527143 (+ 59us) client_negotiation.cc:769] Sending connection context
0928 14:37:58.527237 (+ 94us) client_negotiation.cc:240] Negotiation successful
0928 14:37:58.527358 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:37:58.545578 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:37895
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:37:58.627419 24663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:37:58.627694 24663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:37:58.627736 24663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:37:58.628950 24663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:37:58.629029 24663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:37:58.629062 24663 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 14:37:58.629081 24663 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:37:58.629120 24663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:37:58.630685 24663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.13.126:37895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:37:58.630992 24663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:37:58.631274 24663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:37:58.636330 24668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:58.636394 24669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:37:58.636453 24671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:37:58.636683 24663 server_base.cc:1034] running on GCE node
I20260928 14:37:58.636866 24663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:37:58.637076 24663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:37:58.638213 24663 hybrid_clock.cc:648] HybridClock initialized: now 1790606278638187 us; error 38 us; skew 500 ppm
Sep 28 14:37:58 dist-test-slave-wwwb krb5kdc[24266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606278, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:37:58.640638 24663 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:37:58.641129 24663 webserver.cc:460] Webserver started at http://127.22.13.67:39429/ using document root <none> and password file <none>
I20260928 14:37:58.641324 24663 fs_manager.cc:359] Metadata directory not provided
I20260928 14:37:58.641364 24663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:37:58.641462 24663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:37:58.642568 24663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a"
format_stamp: "Formatted at 2026-09-28 14:37:58 on dist-test-slave-wwwb"
server_key: "f32a19bd98ffca91a479cddfb6f93540"
server_key_iv: "ddfeaab532fa94ec48e1124065cb4e9e"
server_key_version: "encryptionkey@0"
I20260928 14:37:58.642874 24663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a"
format_stamp: "Formatted at 2026-09-28 14:37:58 on dist-test-slave-wwwb"
server_key: "f32a19bd98ffca91a479cddfb6f93540"
server_key_iv: "ddfeaab532fa94ec48e1124065cb4e9e"
server_key_version: "encryptionkey@0"
I20260928 14:37:58.644290 24663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:37:58.644984 24678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:58.645171 24663 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:37:58.645275 24663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a"
format_stamp: "Formatted at 2026-09-28 14:37:58 on dist-test-slave-wwwb"
server_key: "f32a19bd98ffca91a479cddfb6f93540"
server_key_iv: "ddfeaab532fa94ec48e1124065cb4e9e"
server_key_version: "encryptionkey@0"
I20260928 14:37:58.645352 24663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:37:58.664202 24663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:37:58.666194 24663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:37:58.666388 24663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:37:58.666930 24663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:37:58.667024 24663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:58.667073 24663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:37:58.667097 24663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:37:58 dist-test-slave-wwwb krb5kdc[24266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606278, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:58.674626 24663 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:34417
I20260928 14:37:58.674708 24793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:34417 every 8 connection(s)
I20260928 14:37:58.675060 24663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:37:58.677551 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.667544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:48019 (local address 127.22.13.126:37895)
0928 14:37:58.667599 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:58.667605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:58.667838 (+ 233us) server_negotiation.cc:407] Connection header received
0928 14:37:58.668386 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:58.668387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:58.668401 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:58.668465 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:58.669398 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.670186 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.671280 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.671564 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.671632 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.674468 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:58.674470 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:58.674470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:58.674479 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:58.676105 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.676528 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.676529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.676530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.676594 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.676849 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.676850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.676850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.676978 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:58.677064 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:58.677344 (+ 280us) server_negotiation.cc:299] Negotiation successful
0928 14:37:58.677450 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260928 14:37:58.677778 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.667537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:37895 (local address 127.22.13.67:48019)
0928 14:37:58.667715 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:58.667753 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:58.668267 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:58.668544 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:58.668547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:58.668985 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:58.669314 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.669321 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.670298 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.670299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.671168 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.671171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.671683 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.671684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.671866 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.672483 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:58.672494 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:58.674379 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:58.676208 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:58.676214 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:58.676217 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:58.676449 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:58.676659 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:58.676660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:58.676660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:58.676774 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:58.677055 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:58.677061 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:58.677206 (+ 145us) client_negotiation.cc:769] Sending connection context
0928 14:37:58.677317 (+ 111us) client_negotiation.cc:240] Negotiation successful
0928 14:37:58.677540 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
I20260928 14:37:58.678257 24794 heartbeater.cc:347] Connected to a master server at 127.22.13.126:37895
I20260928 14:37:58.678368 24794 heartbeater.cc:464] Registering TS with master...
I20260928 14:37:58.678546 24794 heartbeater.cc:511] Master 127.22.13.126:37895 requested a full tablet report, sending...
I20260928 14:37:58.678988 24309 ts_manager.cc:194] Registered new tserver with Master: 5aa41d15a6fd41d08b67a7a6f9ecb38a (127.22.13.67:34417)
I20260928 14:37:58.679515 24309 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:48019
I20260928 14:37:58.680603 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 24663
I20260928 14:37:58.680711 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:37:58.680912 22581 external_mini_cluster.cc:1428] Setting key d9003397b2d5e0bb8e53e7f59cd31f6a
I20260928 14:37:58.681782 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:37:58.685161 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.679311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:34265 (local address 127.22.13.126:37895)
0928 14:37:58.679354 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:58.679356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:58.679448 (+ 92us) server_negotiation.cc:407] Connection header received
0928 14:37:58.679520 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:58.679521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:58.679529 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:58.679592 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:58.679961 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.680429 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.681429 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.681698 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.681763 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.682788 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:58.682789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:58.682789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:58.682795 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:58.684142 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.684528 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.684529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.684529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.684574 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.684757 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.684757 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.684757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.684847 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:58.684892 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:58.685051 (+ 159us) server_negotiation.cc:299] Negotiation successful
0928 14:37:58.685103 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260928 14:37:58.685304 24800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.679257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:37895 (local address 127.22.13.67:34265)
0928 14:37:58.679397 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:58.679406 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:58.679466 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:58.679594 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:58.679595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:58.679795 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:58.679884 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.679886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.680524 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.680525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.681346 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.681348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.681783 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.681784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.681859 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.682274 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:58.682280 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:58.682700 (+ 420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:58.684222 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:58.684224 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:58.684224 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:58.684454 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:58.684636 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:58.684636 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:58.684637 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:58.684704 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:58.684904 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:58.684905 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:58.684959 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 14:37:58.685033 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 14:37:58.685146 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
Sep 28 14:37:58 dist-test-slave-wwwb krb5kdc[24266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 14:37:58 dist-test-slave-wwwb krb5kdc[24266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:58.705243 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.698469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37806 (local address 127.22.13.126:37895)
0928 14:37:58.698526 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:58.698527 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:58.698535 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:37:58.698562 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:58.698563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:58.698571 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:58.698628 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:58.699182 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.699680 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.700397 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.700591 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.700640 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.702875 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:58.702877 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:58.702877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:58.702885 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:58.704062 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.704436 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.704437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.704437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.704491 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.704813 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.704813 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.704814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.704927 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:58.705009 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:58.705115 (+ 106us) server_negotiation.cc:299] Negotiation successful
0928 14:37:58.705176 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 14:37:58.707053 24309 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:37:58.708624 24309 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:37:58.715257 24814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.711121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41300 (local address 127.22.13.66:38305)
0928 14:37:58.711296 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:58.711302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:58.711517 (+ 215us) server_negotiation.cc:407] Connection header received
0928 14:37:58.711612 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:58.711615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:58.711633 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:58.711704 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:58.711929 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.713038 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.714257 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.714824 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.714926 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.714995 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:58.715113 (+ 118us) server_negotiation.cc:299] Negotiation successful
0928 14:37:58.715153 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":71,"threads_started":1}
I20260928 14:37:58.715257 24812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.710998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:38305 (local address 127.0.0.1:41300)
0928 14:37:58.711169 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:58.711458 (+ 289us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:58.711543 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:58.711768 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:58.711769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:58.711771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:58.711842 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.711844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.713144 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.713145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.714161 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.714164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.714926 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.714927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.715009 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.715023 (+ 14us) client_negotiation.cc:769] Sending connection context
0928 14:37:58.715098 (+ 75us) client_negotiation.cc:240] Negotiation successful
0928 14:37:58.715154 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":2,"thread_start_us":53,"threads_started":1}
I20260928 14:37:58.715394 24811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.711195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38091 (local address 127.0.0.1:45504)
0928 14:37:58.711306 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:58.711349 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:58.711418 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:58.711663 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:58.711666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:58.711669 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:58.711742 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.711744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.713137 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.713138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.714271 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.714272 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.715180 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.715181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.715236 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.715240 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:37:58.715299 (+ 59us) client_negotiation.cc:240] Negotiation successful
0928 14:37:58.715328 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":13,"thread_start_us":68,"threads_started":1}
I20260928 14:37:58.715596 24813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.711110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34874 (local address 127.22.13.67:34417)
0928 14:37:58.711574 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:58.711578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:58.711588 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:37:58.711622 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:58.711624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:58.711638 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:58.711702 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:58.711957 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.712685 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.714195 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.714936 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.715019 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.715082 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:58.715436 (+ 354us) server_negotiation.cc:299] Negotiation successful
0928 14:37:58.715488 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":85,"threads_started":1}
I20260928 14:37:58.715718 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.711308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45504 (local address 127.22.13.65:38091)
0928 14:37:58.711484 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:58.711488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:58.711500 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:37:58.711544 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:58.711547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:58.711562 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:58.711656 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:58.711830 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.713048 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.714334 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.715042 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.715103 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.715152 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:58.715569 (+ 417us) server_negotiation.cc:299] Negotiation successful
0928 14:37:58.715619 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
I20260928 14:37:58.715394 24815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.710998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:34417 (local address 127.0.0.1:34874)
0928 14:37:58.711319 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:58.711351 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:58.711439 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:58.711787 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:58.711788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:58.711790 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:58.711876 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.711878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.712786 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.712787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.714086 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.714089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.715117 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.715118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.715185 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.715188 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:37:58.715234 (+ 46us) client_negotiation.cc:240] Negotiation successful
0928 14:37:58.715303 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":31,"thread_start_us":82,"threads_started":1}
I20260928 14:37:58.716403 24448 tablet_service.cc:1467] Processing CreateTablet for tablet 9e77891b99774f788db175346c3875e2 (DEFAULT_TABLE table=test-table [id=e4c3a29164644e26847d8b085072de2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:37:58.716434 24588 tablet_service.cc:1467] Processing CreateTablet for tablet 9e77891b99774f788db175346c3875e2 (DEFAULT_TABLE table=test-table [id=e4c3a29164644e26847d8b085072de2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:37:58.716435 24728 tablet_service.cc:1467] Processing CreateTablet for tablet 9e77891b99774f788db175346c3875e2 (DEFAULT_TABLE table=test-table [id=e4c3a29164644e26847d8b085072de2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:37:58.716790 24448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e77891b99774f788db175346c3875e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:58.716791 24728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e77891b99774f788db175346c3875e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:58.716795 24588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e77891b99774f788db175346c3875e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:58.719942 24817 tablet_bootstrap.cc:492] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0: Bootstrap starting.
I20260928 14:37:58.720494 24819 tablet_bootstrap.cc:492] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21: Bootstrap starting.
I20260928 14:37:58.720583 24818 tablet_bootstrap.cc:492] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Bootstrap starting.
I20260928 14:37:58.720757 24817 tablet_bootstrap.cc:654] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:58.721128 24817 log.cc:824] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0: Log is configured to *not* fsync() on all Append() calls
I20260928 14:37:58.721165 24819 tablet_bootstrap.cc:654] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:58.721503 24819 log.cc:824] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21: Log is configured to *not* fsync() on all Append() calls
I20260928 14:37:58.721575 24818 tablet_bootstrap.cc:654] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:58.721910 24817 tablet_bootstrap.cc:492] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0: No bootstrap required, opened a new log
I20260928 14:37:58.721928 24818 log.cc:824] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Log is configured to *not* fsync() on all Append() calls
I20260928 14:37:58.722018 24817 ts_tablet_manager.cc:1397] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:37:58.722393 24819 tablet_bootstrap.cc:492] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21: No bootstrap required, opened a new log
I20260928 14:37:58.722463 24819 ts_tablet_manager.cc:1397] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:37:58.722699 24818 tablet_bootstrap.cc:492] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: No bootstrap required, opened a new log
I20260928 14:37:58.722788 24818 ts_tablet_manager.cc:1397] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 14:37:58.724056 24817 raft_consensus.cc:348] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } }
I20260928 14:37:58.724236 24817 raft_consensus.cc:374] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:58.724279 24817 raft_consensus.cc:729] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86bd41d8b4634545a7c4e96e5e50f2d0, State: Initialized, Role: FOLLOWER
I20260928 14:37:58.724363 24818 raft_consensus.cc:348] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } }
I20260928 14:37:58.724426 24817 consensus_queue.cc:260] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } }
I20260928 14:37:58.724536 24818 raft_consensus.cc:374] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:58.724581 24818 raft_consensus.cc:729] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa41d15a6fd41d08b67a7a6f9ecb38a, State: Initialized, Role: FOLLOWER
I20260928 14:37:58.724524 24819 raft_consensus.cc:348] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } }
I20260928 14:37:58.724673 24819 raft_consensus.cc:374] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:58.724711 24819 raft_consensus.cc:729] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abadc8072e634fd98393c935e8ed6d21, State: Initialized, Role: FOLLOWER
I20260928 14:37:58.724740 24817 ts_tablet_manager.cc:1428] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 14:37:58.724689 24818 consensus_queue.cc:260] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } }
I20260928 14:37:58.724848 24514 heartbeater.cc:503] Master 127.22.13.126:37895 was elected leader, sending a full tablet report...
I20260928 14:37:58.724856 24819 consensus_queue.cc:260] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } }
I20260928 14:37:58.724980 24818 ts_tablet_manager.cc:1428] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 14:37:58.725111 24794 heartbeater.cc:503] Master 127.22.13.126:37895 was elected leader, sending a full tablet report...
I20260928 14:37:58.725131 24819 ts_tablet_manager.cc:1428] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:37:58.725257 24654 heartbeater.cc:503] Master 127.22.13.126:37895 was elected leader, sending a full tablet report...
W20260928 14:37:58.767483 24655 tablet.cc:2369] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:37:58.853268 24515 tablet.cc:2369] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:37:58.923139 24823 raft_consensus.cc:482] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:37:58.923259 24823 raft_consensus.cc:504] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } }
I20260928 14:37:58.923599 24823 leader_election.cc:288] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abadc8072e634fd98393c935e8ed6d21 (127.22.13.66:38305), 5aa41d15a6fd41d08b67a7a6f9ecb38a (127.22.13.67:34417)
W20260928 14:37:58.925670 24795 tablet.cc:2369] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:37:58.926915 24813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.923984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:57259 (local address 127.22.13.67:34417)
0928 14:37:58.924044 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:58.924047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:58.924120 (+ 73us) server_negotiation.cc:407] Connection header received
0928 14:37:58.924205 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:58.924206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:58.924216 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:58.924275 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:58.924479 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.924962 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.926181 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.926553 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.926658 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.926666 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:58.926818 (+ 152us) server_negotiation.cc:299] Negotiation successful
0928 14:37:58.926854 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:37:58.926921 24814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.923998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:45743 (local address 127.22.13.66:38305)
0928 14:37:58.924056 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:58.924058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:58.924190 (+ 132us) server_negotiation.cc:407] Connection header received
0928 14:37:58.924229 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:58.924230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:58.924242 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:58.924317 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:58.924528 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.925017 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.926186 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.926542 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.926601 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.926629 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:58.926804 (+ 175us) server_negotiation.cc:299] Negotiation successful
0928 14:37:58.926853 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 14:37:58.926964 24826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.923888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:38305 (local address 127.22.13.65:45743)
0928 14:37:58.924082 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:58.924094 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:58.924179 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:58.924331 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:58.924332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:58.924334 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:58.924421 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.924422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.925107 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.925108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.926089 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.926092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.926646 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.926647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.926718 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.926722 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:37:58.926773 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 14:37:58.926842 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":26,"thread_start_us":59,"threads_started":1}
I20260928 14:37:58.926965 24827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.923888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:34417 (local address 127.22.13.65:57259)
0928 14:37:58.924034 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:58.924046 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:58.924123 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:58.924289 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:58.924290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:58.924293 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:58.924385 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.924387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.925048 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.925049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.926088 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.926091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.926670 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.926671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.926734 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.926737 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:37:58.926805 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 14:37:58.926849 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":45,"thread_start_us":63,"threads_started":1}
I20260928 14:37:58.927588 24608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e77891b99774f788db175346c3875e2" candidate_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abadc8072e634fd98393c935e8ed6d21" is_pre_election: true
I20260928 14:37:58.927588 24748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e77891b99774f788db175346c3875e2" candidate_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" is_pre_election: true
I20260928 14:37:58.927752 24748 raft_consensus.cc:2454] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86bd41d8b4634545a7c4e96e5e50f2d0 in term 0.
I20260928 14:37:58.927752 24608 raft_consensus.cc:2454] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86bd41d8b4634545a7c4e96e5e50f2d0 in term 0.
I20260928 14:37:58.928009 24401 leader_election.cc:302] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86bd41d8b4634545a7c4e96e5e50f2d0, abadc8072e634fd98393c935e8ed6d21; no voters:
I20260928 14:37:58.928174 24823 raft_consensus.cc:2781] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:37:58.928246 24823 raft_consensus.cc:482] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:37:58.928273 24823 raft_consensus.cc:3037] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:58.928972 24823 raft_consensus.cc:504] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } }
I20260928 14:37:58.929131 24823 leader_election.cc:288] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [CANDIDATE]: Term 1 election: Requested vote from peers abadc8072e634fd98393c935e8ed6d21 (127.22.13.66:38305), 5aa41d15a6fd41d08b67a7a6f9ecb38a (127.22.13.67:34417)
I20260928 14:37:58.929324 24748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e77891b99774f788db175346c3875e2" candidate_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a"
I20260928 14:37:58.929339 24608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e77891b99774f788db175346c3875e2" candidate_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abadc8072e634fd98393c935e8ed6d21"
I20260928 14:37:58.929411 24748 raft_consensus.cc:3037] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:58.929412 24608 raft_consensus.cc:3037] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:58.930335 24748 raft_consensus.cc:2454] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86bd41d8b4634545a7c4e96e5e50f2d0 in term 1.
I20260928 14:37:58.930339 24608 raft_consensus.cc:2454] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86bd41d8b4634545a7c4e96e5e50f2d0 in term 1.
I20260928 14:37:58.930510 24401 leader_election.cc:302] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86bd41d8b4634545a7c4e96e5e50f2d0, abadc8072e634fd98393c935e8ed6d21; no voters:
I20260928 14:37:58.930611 24823 raft_consensus.cc:2781] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:37:58.930729 24823 raft_consensus.cc:686] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 LEADER]: Becoming Leader. State: Replica: 86bd41d8b4634545a7c4e96e5e50f2d0, State: Running, Role: LEADER
I20260928 14:37:58.930830 24823 consensus_queue.cc:237] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } }
I20260928 14:37:58.931670 24310 catalog_manager.cc:5495] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86bd41d8b4634545a7c4e96e5e50f2d0 (127.22.13.65). New cstate: current_term: 1 leader_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } health_report { overall_health: HEALTHY } } }
Sep 28 14:37:58 dist-test-slave-wwwb krb5kdc[24266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606278, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:37:58.969658 24835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.962722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:37895 (local address 127.0.0.1:37822)
0928 14:37:58.962898 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:58.962908 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:58.962998 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:58.963166 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:58.963167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:58.963503 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:37:58.963553 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.963556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.964241 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.964243 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.964773 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:58.964775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.965134 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.965135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:58.965186 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.965700 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:37:58.965705 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:37:58.967135 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:37:58.968651 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:58.968659 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:58.968661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:58.968888 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:58.969068 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:37:58.969068 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:37:58.969069 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:37:58.969123 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:37:58.969331 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:37:58.969337 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:37:58.969382 (+ 45us) client_negotiation.cc:769] Sending connection context
0928 14:37:58.969446 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 14:37:58.969544 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":96,"threads_started":1}
I20260928 14:37:58.969734 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:58.962765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37822 (local address 127.22.13.126:37895)
0928 14:37:58.962836 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:58.962839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:58.962979 (+ 140us) server_negotiation.cc:407] Connection header received
0928 14:37:58.963084 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:58.963085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:58.963101 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:58.963162 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:37:58.963645 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.964148 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.964864 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:58.965061 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:58.965135 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:58.967227 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:37:58.967228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:37:58.967229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:37:58.967236 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:37:58.968566 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.968962 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.968963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.968963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.969011 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:37:58.969173 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:37:58.969174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:37:58.969174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:37:58.969280 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:37:58.969327 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:58.969478 (+ 151us) server_negotiation.cc:299] Negotiation successful
0928 14:37:58.969589 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":41}
I20260928 14:37:58.971694 24310 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:37822:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 14:37:58.971935 24310 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:37:58.973528 24448 tablet_service.cc:1467] Processing CreateTablet for tablet a1450ad625234dd9997f049a9370ddf4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbf21358b908438b87c9bf7fcdf1af2f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:37:58.973529 24728 tablet_service.cc:1467] Processing CreateTablet for tablet a1450ad625234dd9997f049a9370ddf4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbf21358b908438b87c9bf7fcdf1af2f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:37:58.973534 24588 tablet_service.cc:1467] Processing CreateTablet for tablet a1450ad625234dd9997f049a9370ddf4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbf21358b908438b87c9bf7fcdf1af2f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:37:58.973713 24448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1450ad625234dd9997f049a9370ddf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:58.973714 24588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1450ad625234dd9997f049a9370ddf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:58.973714 24728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1450ad625234dd9997f049a9370ddf4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:58.975039 24817 tablet_bootstrap.cc:492] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0: Bootstrap starting.
I20260928 14:37:58.975261 24819 tablet_bootstrap.cc:492] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21: Bootstrap starting.
I20260928 14:37:58.975253 24818 tablet_bootstrap.cc:492] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Bootstrap starting.
I20260928 14:37:58.975503 24817 tablet_bootstrap.cc:654] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:58.975699 24818 tablet_bootstrap.cc:654] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:58.975981 24819 tablet_bootstrap.cc:654] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:58.976549 24817 tablet_bootstrap.cc:492] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0: No bootstrap required, opened a new log
I20260928 14:37:58.976594 24817 ts_tablet_manager.cc:1397] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 14:37:58.976678 24819 tablet_bootstrap.cc:492] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21: No bootstrap required, opened a new log
I20260928 14:37:58.976727 24819 ts_tablet_manager.cc:1397] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:37:58.976748 24817 raft_consensus.cc:348] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:58.976820 24817 raft_consensus.cc:374] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:58.976846 24817 raft_consensus.cc:729] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86bd41d8b4634545a7c4e96e5e50f2d0, State: Initialized, Role: FOLLOWER
I20260928 14:37:58.976903 24817 consensus_queue.cc:260] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:58.976889 24819 raft_consensus.cc:348] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:58.976956 24819 raft_consensus.cc:374] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:58.976979 24819 raft_consensus.cc:729] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abadc8072e634fd98393c935e8ed6d21, State: Initialized, Role: FOLLOWER
I20260928 14:37:58.976997 24817 ts_tablet_manager.cc:1428] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:58.977028 24819 consensus_queue.cc:260] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:58.977111 24819 ts_tablet_manager.cc:1428] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:37:58.977082 24823 tablet_replica.cc:439] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } }
I20260928 14:37:58.977165 24823 tablet_replica.cc:442] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:37:58.977206 24818 tablet_bootstrap.cc:492] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: No bootstrap required, opened a new log
I20260928 14:37:58.977252 24818 ts_tablet_manager.cc:1397] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 14:37:58.977238 24825 tablet_replica.cc:439] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } }
I20260928 14:37:58.977311 24825 tablet_replica.cc:442] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:37:58.977368 24818 raft_consensus.cc:348] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:58.977443 24818 raft_consensus.cc:374] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:58.977464 24818 raft_consensus.cc:729] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa41d15a6fd41d08b67a7a6f9ecb38a, State: Initialized, Role: FOLLOWER
I20260928 14:37:58.977515 24818 consensus_queue.cc:260] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:58.977602 24818 ts_tablet_manager.cc:1428] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 14:37:58.977725 24824 tablet_replica.cc:439] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } }
I20260928 14:37:58.977802 24824 tablet_replica.cc:442] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:37:59.001263 24823 raft_consensus.cc:482] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:37:59.001405 24823 raft_consensus.cc:504] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:59.001601 24823 leader_election.cc:288] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abadc8072e634fd98393c935e8ed6d21 (127.22.13.66:38305), 5aa41d15a6fd41d08b67a7a6f9ecb38a (127.22.13.67:34417)
I20260928 14:37:59.001847 24748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1450ad625234dd9997f049a9370ddf4" candidate_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" is_pre_election: true
I20260928 14:37:59.001847 24608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1450ad625234dd9997f049a9370ddf4" candidate_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abadc8072e634fd98393c935e8ed6d21" is_pre_election: true
I20260928 14:37:59.001978 24748 raft_consensus.cc:2454] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86bd41d8b4634545a7c4e96e5e50f2d0 in term 0.
I20260928 14:37:59.001978 24608 raft_consensus.cc:2454] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86bd41d8b4634545a7c4e96e5e50f2d0 in term 0.
I20260928 14:37:59.002183 24400 leader_election.cc:302] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa41d15a6fd41d08b67a7a6f9ecb38a, 86bd41d8b4634545a7c4e96e5e50f2d0; no voters:
I20260928 14:37:59.002302 24823 raft_consensus.cc:2781] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:37:59.002355 24823 raft_consensus.cc:482] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:37:59.002385 24823 raft_consensus.cc:3037] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:59.003072 24823 raft_consensus.cc:504] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:59.003217 24823 leader_election.cc:288] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [CANDIDATE]: Term 1 election: Requested vote from peers abadc8072e634fd98393c935e8ed6d21 (127.22.13.66:38305), 5aa41d15a6fd41d08b67a7a6f9ecb38a (127.22.13.67:34417)
I20260928 14:37:59.003415 24608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1450ad625234dd9997f049a9370ddf4" candidate_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abadc8072e634fd98393c935e8ed6d21"
I20260928 14:37:59.003415 24748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1450ad625234dd9997f049a9370ddf4" candidate_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a"
I20260928 14:37:59.003499 24608 raft_consensus.cc:3037] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:59.003499 24748 raft_consensus.cc:3037] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:59.004266 24608 raft_consensus.cc:2454] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86bd41d8b4634545a7c4e96e5e50f2d0 in term 1.
I20260928 14:37:59.004266 24748 raft_consensus.cc:2454] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86bd41d8b4634545a7c4e96e5e50f2d0 in term 1.
I20260928 14:37:59.004457 24401 leader_election.cc:302] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86bd41d8b4634545a7c4e96e5e50f2d0, abadc8072e634fd98393c935e8ed6d21; no voters:
I20260928 14:37:59.004583 24823 raft_consensus.cc:2781] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:37:59.004676 24823 raft_consensus.cc:686] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 LEADER]: Becoming Leader. State: Replica: 86bd41d8b4634545a7c4e96e5e50f2d0, State: Running, Role: LEADER
I20260928 14:37:59.004731 24823 consensus_queue.cc:237] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:59.005067 24828 tablet_replica.cc:439] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0: TxnStatusTablet state changed. Reason: New leader 86bd41d8b4634545a7c4e96e5e50f2d0. Latest consensus state: current_term: 1 leader_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } }
I20260928 14:37:59.005164 24828 tablet_replica.cc:442] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:37:59.005399 24309 catalog_manager.cc:5495] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86bd41d8b4634545a7c4e96e5e50f2d0 (127.22.13.65). New cstate: current_term: 1 leader_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } health_report { overall_health: UNKNOWN } } }
I20260928 14:37:59.005699 24840 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 14:37:59.005863 24840 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 14:37:59.016960 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:59.014099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45512 (local address 127.22.13.65:38091)
0928 14:37:59.014162 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:59.014164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:59.014251 (+ 87us) server_negotiation.cc:407] Connection header received
0928 14:37:59.014347 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:59.014347 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:59.014364 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:59.014426 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:37:59.014598 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.015073 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.016186 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.016383 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.016435 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:59.016651 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:37:59.016751 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:37:59.016804 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:59.016878 (+ 74us) server_negotiation.cc:299] Negotiation successful
0928 14:37:59.016898 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:37:59.017011 24835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:59.014034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38091 (local address 127.0.0.1:45512)
0928 14:37:59.014178 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:59.014192 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:59.014264 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:59.014461 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:59.014462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:59.014464 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:37:59.014526 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:59.014528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.015148 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.015149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:59.016099 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:59.016102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.016462 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.016463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:59.016533 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:59.016552 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:37:59.016807 (+ 255us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:37:59.016815 (+ 8us) client_negotiation.cc:769] Sending connection context
0928 14:37:59.016872 (+ 57us) client_negotiation.cc:240] Negotiation successful
0928 14:37:59.016925 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260928 14:37:59.019007 24748 raft_consensus.cc:1261] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Refusing update from remote peer 86bd41d8b4634545a7c4e96e5e50f2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:37:59.019021 24608 raft_consensus.cc:1261] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Refusing update from remote peer 86bd41d8b4634545a7c4e96e5e50f2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:37:59.019356 24823 consensus_queue.cc:1035] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:37:59.019510 24828 consensus_queue.cc:1035] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:37:59.020776 24825 tablet_replica.cc:439] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21: TxnStatusTablet state changed. Reason: New leader 86bd41d8b4634545a7c4e96e5e50f2d0. Latest consensus state: current_term: 1 leader_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } }
I20260928 14:37:59.020787 24824 tablet_replica.cc:439] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: TxnStatusTablet state changed. Reason: New leader 86bd41d8b4634545a7c4e96e5e50f2d0. Latest consensus state: current_term: 1 leader_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } }
I20260928 14:37:59.020877 24825 tablet_replica.cc:442] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:37:59.020893 24824 tablet_replica.cc:442] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:37:59.022218 24828 tablet_replica.cc:439] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } }
I20260928 14:37:59.022323 24828 tablet_replica.cc:442] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:37:59.022222 24843 mvcc.cc:205] Tried to move back new op lower bound from 7334323318859153408 to 7334323318803787776. Current Snapshot: MvccSnapshot[applied={T|T < 7334323318859153408}]
I20260928 14:37:59.022581 24828 tablet_replica.cc:439] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } }
I20260928 14:37:59.022581 24825 tablet_replica.cc:439] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } }
I20260928 14:37:59.022651 24828 tablet_replica.cc:442] T a1450ad625234dd9997f049a9370ddf4 P 86bd41d8b4634545a7c4e96e5e50f2d0: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:37:59.022663 24825 tablet_replica.cc:442] T a1450ad625234dd9997f049a9370ddf4 P abadc8072e634fd98393c935e8ed6d21: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:37:59.023254 24824 tablet_replica.cc:439] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } } }
I20260928 14:37:59.023348 24824 tablet_replica.cc:442] T a1450ad625234dd9997f049a9370ddf4 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:37:59.023353 24844 mvcc.cc:205] Tried to move back new op lower bound from 7334323318859153408 to 7334323318803787776. Current Snapshot: MvccSnapshot[applied={T|T < 7334323318859153408 or (T in {7334323318859153408})}]
I20260928 14:37:59.023640 24845 mvcc.cc:205] Tried to move back new op lower bound from 7334323318859153408 to 7334323318803787776. Current Snapshot: MvccSnapshot[applied={T|T < 7334323318859153408}]
I20260928 14:37:59.027846 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:59.025473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45522 (local address 127.22.13.65:38091)
0928 14:37:59.025521 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:59.025523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:59.025532 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:37:59.025567 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:59.025568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:59.025579 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:59.025632 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:37:59.025790 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.026333 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.027141 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.027345 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.027429 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:59.027581 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:37:59.027617 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:37:59.027669 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:59.027731 (+ 62us) server_negotiation.cc:299] Negotiation successful
0928 14:37:59.027756 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 14:37:59.033277 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:59.030402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:54081 (local address 127.22.13.65:38091)
0928 14:37:59.030445 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:59.030447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:59.030610 (+ 163us) server_negotiation.cc:407] Connection header received
0928 14:37:59.030689 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:59.030690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:59.030697 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:59.030736 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:37:59.030878 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.031304 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.032458 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.032629 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.032675 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:59.032938 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:37:59.032972 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:37:59.033025 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:59.033168 (+ 143us) server_negotiation.cc:299] Negotiation successful
0928 14:37:59.033193 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":23}
I20260928 14:37:59.033377 24855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:59.030327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38091 (local address 127.22.13.65:54081)
0928 14:37:59.030524 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:59.030537 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:59.030620 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:59.030748 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:59.030749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:59.030751 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:37:59.030821 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:59.030823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.031371 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.031372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:59.032365 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:59.032368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.032695 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.032697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:59.032764 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:59.032782 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:37:59.033051 (+ 269us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:37:59.033053 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:37:59.033161 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 14:37:59.033228 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":91,"threads_started":1}
I20260928 14:37:59.035526 24748 raft_consensus.cc:1261] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Refusing update from remote peer 86bd41d8b4634545a7c4e96e5e50f2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:37:59.035544 24608 raft_consensus.cc:1261] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Refusing update from remote peer 86bd41d8b4634545a7c4e96e5e50f2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:37:59.035754 24849 consensus_queue.cc:1035] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:37:59.035883 24823 consensus_queue.cc:1035] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:37:59.047832 24309 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:37:59.047942 24309 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:37:59.048804 24309 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7: Sending DeleteTablet for 3 replicas of tablet 9e77891b99774f788db175346c3875e2
I20260928 14:37:59.049114 24588 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e77891b99774f788db175346c3875e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:37:59 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:41300
I20260928 14:37:59.049270 24447 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e77891b99774f788db175346c3875e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:37:59 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:45504
I20260928 14:37:59.049345 24728 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e77891b99774f788db175346c3875e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:37:59 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:34874
I20260928 14:37:59.049563 24861 tablet_replica.cc:330] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: stopping tablet replica
I20260928 14:37:59.049701 24861 raft_consensus.cc:2229] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:37:59.049677 24309 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260928 14:37:59.049813 24861 raft_consensus.cc:2258] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260928 14:37:59.049875 24309 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:37:59.049719 24860 tablet_replica.cc:330] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0: stopping tablet replica
I20260928 14:37:59.050175 24860 raft_consensus.cc:2229] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:37:59.050340 24860 raft_consensus.cc:2258] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:37:59.050392 24861 ts_tablet_manager.cc:1905] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:37:59.050828 24860 ts_tablet_manager.cc:1905] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:37:59.051553 24447 tablet_service.cc:1467] Processing CreateTablet for tablet b0d4abc4f9324b0d9e19027ec92ba646 (DEFAULT_TABLE table=test-table [id=818ba496712f48e99838b72b2bf8599b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:37:59.051683 24447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0d4abc4f9324b0d9e19027ec92ba646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:59.052274 24861 ts_tablet_manager.cc:1918] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:37:59.052378 24861 log.cc:1196] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/9e77891b99774f788db175346c3875e2
I20260928 14:37:59.052664 24861 ts_tablet_manager.cc:1939] T 9e77891b99774f788db175346c3875e2 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Deleting consensus metadata
I20260928 14:37:59.052882 24860 ts_tablet_manager.cc:1918] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:37:59.053007 24860 log.cc:1196] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/wal/wals/9e77891b99774f788db175346c3875e2
I20260928 14:37:59.053154 24817 tablet_bootstrap.cc:492] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0: Bootstrap starting.
I20260928 14:37:59.053292 24860 ts_tablet_manager.cc:1939] T 9e77891b99774f788db175346c3875e2 P 86bd41d8b4634545a7c4e96e5e50f2d0: Deleting consensus metadata
I20260928 14:37:59.053561 24817 tablet_bootstrap.cc:654] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:59.053990 24296 catalog_manager.cc:4841] TS 5aa41d15a6fd41d08b67a7a6f9ecb38a (127.22.13.67:34417): tablet 9e77891b99774f788db175346c3875e2 (table test-table [id=e4c3a29164644e26847d8b085072de2d]) successfully deleted
I20260928 14:37:59.054111 24296 catalog_manager.cc:4841] TS 86bd41d8b4634545a7c4e96e5e50f2d0 (127.22.13.65:38091): tablet 9e77891b99774f788db175346c3875e2 (table test-table [id=e4c3a29164644e26847d8b085072de2d]) successfully deleted
I20260928 14:37:59.054108 24728 tablet_service.cc:1467] Processing CreateTablet for tablet b0d4abc4f9324b0d9e19027ec92ba646 (DEFAULT_TABLE table=test-table [id=818ba496712f48e99838b72b2bf8599b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:37:59.054222 24728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0d4abc4f9324b0d9e19027ec92ba646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:59.054229 24817 tablet_bootstrap.cc:492] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0: No bootstrap required, opened a new log
I20260928 14:37:59.054270 24817 ts_tablet_manager.cc:1397] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:37:59.054414 24817 raft_consensus.cc:348] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:59.054507 24817 raft_consensus.cc:374] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:59.054528 24817 raft_consensus.cc:729] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86bd41d8b4634545a7c4e96e5e50f2d0, State: Initialized, Role: FOLLOWER
I20260928 14:37:59.054575 24817 consensus_queue.cc:260] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:59.054656 24817 ts_tablet_manager.cc:1428] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 14:37:59.055461 24818 tablet_bootstrap.cc:492] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Bootstrap starting.
I20260928 14:37:59.055611 24588 tablet_service.cc:1467] Processing CreateTablet for tablet b0d4abc4f9324b0d9e19027ec92ba646 (DEFAULT_TABLE table=test-table [id=818ba496712f48e99838b72b2bf8599b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:37:59.055755 24588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0d4abc4f9324b0d9e19027ec92ba646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:37:59.055830 24818 tablet_bootstrap.cc:654] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:59.056492 24818 tablet_bootstrap.cc:492] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: No bootstrap required, opened a new log
I20260928 14:37:59.056543 24818 ts_tablet_manager.cc:1397] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:37:59.056640 24859 tablet_replica.cc:330] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21: stopping tablet replica
I20260928 14:37:59.056694 24818 raft_consensus.cc:348] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:59.056761 24818 raft_consensus.cc:374] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:59.056770 24859 raft_consensus.cc:2229] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:37:59.056787 24818 raft_consensus.cc:729] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa41d15a6fd41d08b67a7a6f9ecb38a, State: Initialized, Role: FOLLOWER
I20260928 14:37:59.056833 24859 raft_consensus.cc:2258] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:37:59.056828 24818 consensus_queue.cc:260] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:59.056927 24818 ts_tablet_manager.cc:1428] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 14:37:59.057120 24819 tablet_bootstrap.cc:492] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21: Bootstrap starting.
I20260928 14:37:59.057535 24819 tablet_bootstrap.cc:654] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21: Neither blocks nor log segments found. Creating new log.
I20260928 14:37:59.058171 24819 tablet_bootstrap.cc:492] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21: No bootstrap required, opened a new log
I20260928 14:37:59.058223 24819 ts_tablet_manager.cc:1397] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:37:59.058311 24859 ts_tablet_manager.cc:1905] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:37:59.058398 24819 raft_consensus.cc:348] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:59.058475 24819 raft_consensus.cc:374] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:37:59.058498 24819 raft_consensus.cc:729] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abadc8072e634fd98393c935e8ed6d21, State: Initialized, Role: FOLLOWER
I20260928 14:37:59.058550 24819 consensus_queue.cc:260] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:59.058637 24819 ts_tablet_manager.cc:1428] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 14:37:59.059799 24859 ts_tablet_manager.cc:1918] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:37:59.059876 24859 log.cc:1196] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/9e77891b99774f788db175346c3875e2
I20260928 14:37:59.060127 24859 ts_tablet_manager.cc:1939] T 9e77891b99774f788db175346c3875e2 P abadc8072e634fd98393c935e8ed6d21: Deleting consensus metadata
I20260928 14:37:59.060520 24297 catalog_manager.cc:4841] TS abadc8072e634fd98393c935e8ed6d21 (127.22.13.66:38305): tablet 9e77891b99774f788db175346c3875e2 (table test-table [id=e4c3a29164644e26847d8b085072de2d]) successfully deleted
I20260928 14:37:59.087693 24824 raft_consensus.cc:482] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:37:59.087823 24824 raft_consensus.cc:504] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:59.088187 24824 leader_election.cc:288] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86bd41d8b4634545a7c4e96e5e50f2d0 (127.22.13.65:38091), abadc8072e634fd98393c935e8ed6d21 (127.22.13.66:38305)
I20260928 14:37:59.091596 24865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:59.088342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38091 (local address 127.22.13.67:44093)
0928 14:37:59.088741 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:59.088751 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:59.088822 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:59.088953 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:59.088954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:59.088955 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:59.089027 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:59.089028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.089642 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.089643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:59.090725 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:59.090728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.091300 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.091301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:59.091380 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:59.091384 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:37:59.091451 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 14:37:59.091493 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":192,"thread_start_us":102,"threads_started":1}
I20260928 14:37:59.091576 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:59.088428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:44093 (local address 127.22.13.65:38091)
0928 14:37:59.088533 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:59.088535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:59.088799 (+ 264us) server_negotiation.cc:407] Connection header received
0928 14:37:59.088887 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:59.088888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:59.088896 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:59.088951 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:59.089107 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.089558 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.090824 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.091202 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.091290 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:59.091303 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:59.091458 (+ 155us) server_negotiation.cc:299] Negotiation successful
0928 14:37:59.091497 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260928 14:37:59.091904 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:59.088339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:38305 (local address 127.22.13.67:49371)
0928 14:37:59.088410 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:59.088422 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:59.088470 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:59.088634 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:59.088635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:59.088637 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:37:59.088718 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:59.088721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.089630 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.089632 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:59.090836 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:59.090839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.091631 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.091632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:59.091721 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:59.091725 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:37:59.091794 (+ 69us) client_negotiation.cc:240] Negotiation successful
0928 14:37:59.091834 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260928 14:37:59.091931 24814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:59.088413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:49371 (local address 127.22.13.66:38305)
0928 14:37:59.088470 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:59.088472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:59.088480 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:37:59.088549 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:59.088550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:59.088562 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:59.088630 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:37:59.088784 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.089522 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.090938 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.091511 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.091589 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:59.091598 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:59.091800 (+ 202us) server_negotiation.cc:299] Negotiation successful
0928 14:37:59.091855 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:37:59.092172 24608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0d4abc4f9324b0d9e19027ec92ba646" candidate_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abadc8072e634fd98393c935e8ed6d21" is_pre_election: true
I20260928 14:37:59.092276 24608 raft_consensus.cc:2454] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aa41d15a6fd41d08b67a7a6f9ecb38a in term 0.
I20260928 14:37:59.092260 24468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0d4abc4f9324b0d9e19027ec92ba646" candidate_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" is_pre_election: true
I20260928 14:37:59.092418 24468 raft_consensus.cc:2454] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aa41d15a6fd41d08b67a7a6f9ecb38a in term 0.
I20260928 14:37:59.092514 24681 leader_election.cc:302] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa41d15a6fd41d08b67a7a6f9ecb38a, abadc8072e634fd98393c935e8ed6d21; no voters:
I20260928 14:37:59.092653 24824 raft_consensus.cc:2781] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:37:59.092694 24824 raft_consensus.cc:482] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:37:59.092716 24824 raft_consensus.cc:3037] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:59.093621 24824 raft_consensus.cc:504] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:59.093777 24824 leader_election.cc:288] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [CANDIDATE]: Term 1 election: Requested vote from peers 86bd41d8b4634545a7c4e96e5e50f2d0 (127.22.13.65:38091), abadc8072e634fd98393c935e8ed6d21 (127.22.13.66:38305)
I20260928 14:37:59.094003 24468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0d4abc4f9324b0d9e19027ec92ba646" candidate_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0"
I20260928 14:37:59.094084 24468 raft_consensus.cc:3037] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:59.094038 24608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0d4abc4f9324b0d9e19027ec92ba646" candidate_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abadc8072e634fd98393c935e8ed6d21"
I20260928 14:37:59.094118 24608 raft_consensus.cc:3037] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:37:59.094815 24468 raft_consensus.cc:2454] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aa41d15a6fd41d08b67a7a6f9ecb38a in term 1.
I20260928 14:37:59.094897 24608 raft_consensus.cc:2454] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aa41d15a6fd41d08b67a7a6f9ecb38a in term 1.
I20260928 14:37:59.095024 24680 leader_election.cc:302] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa41d15a6fd41d08b67a7a6f9ecb38a, 86bd41d8b4634545a7c4e96e5e50f2d0; no voters:
I20260928 14:37:59.095134 24824 raft_consensus.cc:2781] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:37:59.095309 24824 raft_consensus.cc:686] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 LEADER]: Becoming Leader. State: Replica: 5aa41d15a6fd41d08b67a7a6f9ecb38a, State: Running, Role: LEADER
I20260928 14:37:59.095388 24824 consensus_queue.cc:237] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:37:59.096119 24309 catalog_manager.cc:5495] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5aa41d15a6fd41d08b67a7a6f9ecb38a (127.22.13.67). New cstate: current_term: 1 leader_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } health_report { overall_health: HEALTHY } } }
I20260928 14:37:59.110522 24813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:59.107893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34890 (local address 127.22.13.67:34417)
0928 14:37:59.107956 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:59.107958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:59.107968 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:37:59.108003 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:59.108004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:59.108025 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:59.108112 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:37:59.108196 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.108916 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.109637 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.109915 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.110029 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:59.110163 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:37:59.110300 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:37:59.110383 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:59.110403 (+ 20us) server_negotiation.cc:299] Negotiation successful
0928 14:37:59.110425 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:37:59.115607 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:59.113182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:44161 (local address 127.22.13.65:38091)
0928 14:37:59.113240 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:59.113242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:59.113255 (+ 13us) server_negotiation.cc:407] Connection header received
0928 14:37:59.113283 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:59.113284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:59.113292 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:59.113336 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:37:59.113464 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.113918 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.114977 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.115151 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.115200 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:59.115396 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:37:59.115423 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:37:59.115459 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:59.115521 (+ 62us) server_negotiation.cc:299] Negotiation successful
0928 14:37:59.115535 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:37:59.115653 24800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:59.113108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38091 (local address 127.22.13.67:44161)
0928 14:37:59.113183 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:59.113193 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:59.113232 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:59.113339 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:59.113340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:59.113341 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:37:59.113403 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:59.113404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.114028 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.114029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:59.114893 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:59.114896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.115224 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.115225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:59.115297 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:59.115315 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:37:59.115463 (+ 148us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:37:59.115464 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 14:37:59.115517 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 14:37:59.115564 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260928 14:37:59.117813 24608 raft_consensus.cc:1261] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Refusing update from remote peer 5aa41d15a6fd41d08b67a7a6f9ecb38a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:37:59.117916 24468 raft_consensus.cc:1261] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 FOLLOWER]: Refusing update from remote peer 5aa41d15a6fd41d08b67a7a6f9ecb38a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:37:59.118088 24824 consensus_queue.cc:1035] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:37:59.118216 24866 consensus_queue.cc:1035] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [LEADER]: Connected to new peer: Peer: permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:37:59.125896 24813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:59.123227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:54769 (local address 127.22.13.67:34417)
0928 14:37:59.123270 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:37:59.123272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:37:59.123282 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:37:59.123353 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:37:59.123354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:37:59.123364 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:37:59.123424 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:37:59.123564 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.124336 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.125099 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.125402 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.125477 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:59.125655 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:37:59.125693 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:37:59.125741 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 14:37:59.125819 (+ 78us) server_negotiation.cc:299] Negotiation successful
0928 14:37:59.125838 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:37:59.125988 24855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:37:59.123145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:34417 (local address 127.22.13.65:54769)
0928 14:37:59.123234 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0928 14:37:59.123249 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:37:59.123286 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:37:59.123418 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:37:59.123419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:37:59.123421 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:37:59.123485 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:59.123487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.124424 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.124425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:59.124958 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:37:59.124960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:37:59.125513 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:37:59.125514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:37:59.125577 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:37:59.125584 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:37:59.125760 (+ 176us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:37:59.125762 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:37:59.125819 (+ 57us) client_negotiation.cc:240] Negotiation successful
0928 14:37:59.125882 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
W20260928 14:37:59.126287 24845 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260928 14:38:00.648077 24309 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:00.648228 24309 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:00.649089 24309 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7: Sending DeleteTablet for 3 replicas of tablet b0d4abc4f9324b0d9e19027ec92ba646
I20260928 14:38:00.649380 24588 tablet_service.cc:1514] Processing DeleteTablet for tablet b0d4abc4f9324b0d9e19027ec92ba646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:00 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:41300
I20260928 14:38:00.649375 24447 tablet_service.cc:1514] Processing DeleteTablet for tablet b0d4abc4f9324b0d9e19027ec92ba646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:00 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:45504
I20260928 14:38:00.649617 24895 tablet_replica.cc:330] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21: stopping tablet replica
I20260928 14:38:00.649632 24728 tablet_service.cc:1514] Processing DeleteTablet for tablet b0d4abc4f9324b0d9e19027ec92ba646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:00 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:34874
I20260928 14:38:00.649694 24895 raft_consensus.cc:2229] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:00.649744 24895 raft_consensus.cc:2258] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:00.649852 24897 tablet_replica.cc:330] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: stopping tablet replica
I20260928 14:38:00.650050 24895 ts_tablet_manager.cc:1905] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:00.652081 24896 tablet_replica.cc:330] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0: stopping tablet replica
I20260928 14:38:00.652171 24896 raft_consensus.cc:2229] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:00.652226 24896 raft_consensus.cc:2258] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:00.652534 24896 ts_tablet_manager.cc:1905] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:00.649925 24897 raft_consensus.cc:2229] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:38:00.652829 24897 raft_consensus.cc:2258] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:00.653079 24897 ts_tablet_manager.cc:1905] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 28 14:38:00 dist-test-slave-wwwb krb5kdc[24266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606280, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260928 14:38:00.681159 24895 ts_tablet_manager.cc:1918] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 14:38:00.681252 24895 log.cc:1196] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/b0d4abc4f9324b0d9e19027ec92ba646
I20260928 14:38:00.681523 24895 ts_tablet_manager.cc:1939] T b0d4abc4f9324b0d9e19027ec92ba646 P abadc8072e634fd98393c935e8ed6d21: Deleting consensus metadata
I20260928 14:38:00.681993 24297 catalog_manager.cc:4841] TS abadc8072e634fd98393c935e8ed6d21 (127.22.13.66:38305): tablet b0d4abc4f9324b0d9e19027ec92ba646 (table test-table [id=818ba496712f48e99838b72b2bf8599b]) successfully deleted
Sep 28 14:38:00 dist-test-slave-wwwb krb5kdc[24266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606280, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:00.706727 24905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:00.695408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37838 (local address 127.22.13.126:37895)
0928 14:38:00.695538 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:00.695540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:00.695553 (+ 13us) server_negotiation.cc:407] Connection header received
0928 14:38:00.695634 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:00.695634 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:00.695646 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:00.695718 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:00.696532 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:00.697048 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:00.698348 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:00.698564 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:00.698626 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:00.704044 (+ 5418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:00.704045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:00.704046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:00.704054 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:00.705660 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:00.706036 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:00.706038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:00.706038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:00.706106 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:00.706282 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:00.706283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:00.706283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:00.706412 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:00.706487 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:00.706582 (+ 95us) server_negotiation.cc:299] Negotiation successful
0928 14:38:00.706652 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
I20260928 14:38:00.708380 24309 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:38:00.708607 24309 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:38:00.710013 24447 tablet_service.cc:1467] Processing CreateTablet for tablet 191daa038cc24e74a9b2fb562471fbd8 (DEFAULT_TABLE table=test-table [id=920eb916c2d1444c96de6be569355d07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:00.710069 24896 ts_tablet_manager.cc:1918] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 14:38:00.710052 24728 tablet_service.cc:1467] Processing CreateTablet for tablet 191daa038cc24e74a9b2fb562471fbd8 (DEFAULT_TABLE table=test-table [id=920eb916c2d1444c96de6be569355d07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:00.710116 24896 log.cc:1196] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/wal/wals/b0d4abc4f9324b0d9e19027ec92ba646
I20260928 14:38:00.710151 24447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 191daa038cc24e74a9b2fb562471fbd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:00.710304 24588 tablet_service.cc:1467] Processing CreateTablet for tablet 191daa038cc24e74a9b2fb562471fbd8 (DEFAULT_TABLE table=test-table [id=920eb916c2d1444c96de6be569355d07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:00.710372 24896 ts_tablet_manager.cc:1939] T b0d4abc4f9324b0d9e19027ec92ba646 P 86bd41d8b4634545a7c4e96e5e50f2d0: Deleting consensus metadata
I20260928 14:38:00.710422 24588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 191daa038cc24e74a9b2fb562471fbd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:00.711805 24906 tablet_bootstrap.cc:492] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21: Bootstrap starting.
I20260928 14:38:00.711864 24907 tablet_bootstrap.cc:492] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0: Bootstrap starting.
I20260928 14:38:00.711902 24897 ts_tablet_manager.cc:1918] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 14:38:00.711957 24897 log.cc:1196] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/b0d4abc4f9324b0d9e19027ec92ba646
I20260928 14:38:00.712184 24906 tablet_bootstrap.cc:654] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:00.712241 24907 tablet_bootstrap.cc:654] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:00.712265 24897 ts_tablet_manager.cc:1939] T b0d4abc4f9324b0d9e19027ec92ba646 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Deleting consensus metadata
I20260928 14:38:00.712430 24728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 191daa038cc24e74a9b2fb562471fbd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:00.712787 24906 tablet_bootstrap.cc:492] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21: No bootstrap required, opened a new log
I20260928 14:38:00.712829 24906 ts_tablet_manager.cc:1397] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:00.712849 24907 tablet_bootstrap.cc:492] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0: No bootstrap required, opened a new log
I20260928 14:38:00.712886 24907 ts_tablet_manager.cc:1397] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:00.712980 24907 raft_consensus.cc:348] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:00.713032 24907 raft_consensus.cc:374] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:00.712986 24906 raft_consensus.cc:348] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:00.713047 24907 raft_consensus.cc:729] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86bd41d8b4634545a7c4e96e5e50f2d0, State: Initialized, Role: FOLLOWER
I20260928 14:38:00.713054 24906 raft_consensus.cc:374] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:00.713083 24906 raft_consensus.cc:729] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abadc8072e634fd98393c935e8ed6d21, State: Initialized, Role: FOLLOWER
I20260928 14:38:00.713115 24907 consensus_queue.cc:260] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:00.713129 24906 consensus_queue.cc:260] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:00.713294 24906 ts_tablet_manager.cc:1428] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:00.713452 24907 ts_tablet_manager.cc:1428] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:00.714025 24911 tablet_bootstrap.cc:492] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Bootstrap starting.
I20260928 14:38:00.714447 24911 tablet_bootstrap.cc:654] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:00.715016 24911 tablet_bootstrap.cc:492] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: No bootstrap required, opened a new log
I20260928 14:38:00.715083 24911 ts_tablet_manager.cc:1397] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:00.715214 24911 raft_consensus.cc:348] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:00.715296 24911 raft_consensus.cc:374] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:00.715327 24911 raft_consensus.cc:729] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa41d15a6fd41d08b67a7a6f9ecb38a, State: Initialized, Role: FOLLOWER
I20260928 14:38:00.715386 24911 consensus_queue.cc:260] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:00.715489 24911 ts_tablet_manager.cc:1428] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:00.729529 24296 catalog_manager.cc:4841] TS 86bd41d8b4634545a7c4e96e5e50f2d0 (127.22.13.65:38091): tablet b0d4abc4f9324b0d9e19027ec92ba646 (table test-table [id=818ba496712f48e99838b72b2bf8599b]) successfully deleted
I20260928 14:38:00.729640 24296 catalog_manager.cc:4841] TS 5aa41d15a6fd41d08b67a7a6f9ecb38a (127.22.13.67:34417): tablet b0d4abc4f9324b0d9e19027ec92ba646 (table test-table [id=818ba496712f48e99838b72b2bf8599b]) successfully deleted
I20260928 14:38:00.947139 24876 raft_consensus.cc:482] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:38:00.947244 24876 raft_consensus.cc:504] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:00.947427 24876 leader_election.cc:288] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abadc8072e634fd98393c935e8ed6d21 (127.22.13.66:38305), 5aa41d15a6fd41d08b67a7a6f9ecb38a (127.22.13.67:34417)
I20260928 14:38:00.947678 24748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "191daa038cc24e74a9b2fb562471fbd8" candidate_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" is_pre_election: true
I20260928 14:38:00.947678 24608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "191daa038cc24e74a9b2fb562471fbd8" candidate_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abadc8072e634fd98393c935e8ed6d21" is_pre_election: true
I20260928 14:38:00.947777 24748 raft_consensus.cc:2454] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86bd41d8b4634545a7c4e96e5e50f2d0 in term 0.
I20260928 14:38:00.947777 24608 raft_consensus.cc:2454] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86bd41d8b4634545a7c4e96e5e50f2d0 in term 0.
I20260928 14:38:00.947945 24401 leader_election.cc:302] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86bd41d8b4634545a7c4e96e5e50f2d0, abadc8072e634fd98393c935e8ed6d21; no voters:
I20260928 14:38:00.948051 24876 raft_consensus.cc:2781] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:38:00.948092 24876 raft_consensus.cc:482] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:38:00.948153 24876 raft_consensus.cc:3037] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:00.992861 24876 raft_consensus.cc:504] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:00.993099 24876 leader_election.cc:288] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [CANDIDATE]: Term 1 election: Requested vote from peers abadc8072e634fd98393c935e8ed6d21 (127.22.13.66:38305), 5aa41d15a6fd41d08b67a7a6f9ecb38a (127.22.13.67:34417)
I20260928 14:38:00.993323 24608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "191daa038cc24e74a9b2fb562471fbd8" candidate_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abadc8072e634fd98393c935e8ed6d21"
I20260928 14:38:00.993323 24748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "191daa038cc24e74a9b2fb562471fbd8" candidate_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a"
I20260928 14:38:00.993432 24608 raft_consensus.cc:3037] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:00.993432 24748 raft_consensus.cc:3037] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:00.994290 24608 raft_consensus.cc:2454] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86bd41d8b4634545a7c4e96e5e50f2d0 in term 1.
I20260928 14:38:00.994290 24748 raft_consensus.cc:2454] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86bd41d8b4634545a7c4e96e5e50f2d0 in term 1.
I20260928 14:38:00.994499 24400 leader_election.cc:302] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa41d15a6fd41d08b67a7a6f9ecb38a, 86bd41d8b4634545a7c4e96e5e50f2d0; no voters:
I20260928 14:38:00.994638 24876 raft_consensus.cc:2781] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:00.994691 24876 raft_consensus.cc:686] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 LEADER]: Becoming Leader. State: Replica: 86bd41d8b4634545a7c4e96e5e50f2d0, State: Running, Role: LEADER
I20260928 14:38:00.994745 24876 consensus_queue.cc:237] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:00.995388 24310 catalog_manager.cc:5495] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86bd41d8b4634545a7c4e96e5e50f2d0 (127.22.13.65). New cstate: current_term: 1 leader_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } health_report { overall_health: UNKNOWN } } }
I20260928 14:38:01.069268 24915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:01.067034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45532 (local address 127.22.13.65:38091)
0928 14:38:01.067185 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:01.067187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:01.067196 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:38:01.067226 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:01.067227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:01.067236 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:01.067290 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:38:01.067416 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.067898 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.068628 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.068815 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.068866 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:01.069010 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:38:01.069037 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:38:01.069076 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:01.069132 (+ 56us) server_negotiation.cc:299] Negotiation successful
0928 14:38:01.069150 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260928 14:38:01.071638 24608 raft_consensus.cc:1261] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Refusing update from remote peer 86bd41d8b4634545a7c4e96e5e50f2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:01.071658 24748 raft_consensus.cc:1261] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Refusing update from remote peer 86bd41d8b4634545a7c4e96e5e50f2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:01.071863 24849 consensus_queue.cc:1035] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:01.072005 24876 consensus_queue.cc:1035] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:01.081854 24309 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:01.082010 24309 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:01.082655 24309 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7: Sending DeleteTablet for 3 replicas of tablet 191daa038cc24e74a9b2fb562471fbd8
I20260928 14:38:01.082933 24588 tablet_service.cc:1514] Processing DeleteTablet for tablet 191daa038cc24e74a9b2fb562471fbd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:01 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:41300
I20260928 14:38:01.082945 24728 tablet_service.cc:1514] Processing DeleteTablet for tablet 191daa038cc24e74a9b2fb562471fbd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:01 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:34874
I20260928 14:38:01.083042 24895 tablet_replica.cc:330] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21: stopping tablet replica
I20260928 14:38:01.083058 24897 tablet_replica.cc:330] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: stopping tablet replica
I20260928 14:38:01.083101 24897 raft_consensus.cc:2229] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:01.083101 24895 raft_consensus.cc:2229] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:01.083133 24897 raft_consensus.cc:2258] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:01.083148 24895 raft_consensus.cc:2258] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:01.083268 24447 tablet_service.cc:1514] Processing DeleteTablet for tablet 191daa038cc24e74a9b2fb562471fbd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:01 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:45504
I20260928 14:38:01.083351 24896 tablet_replica.cc:330] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0: stopping tablet replica
I20260928 14:38:01.083424 24896 raft_consensus.cc:2229] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:38:01.083367 24309 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260928 14:38:01.083498 24896 raft_consensus.cc:2258] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260928 14:38:01.083508 24309 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:38:01.083596 24895 ts_tablet_manager.cc:1905] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:01.083788 24896 ts_tablet_manager.cc:1905] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:01.083807 24897 ts_tablet_manager.cc:1905] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:01.085276 24588 tablet_service.cc:1467] Processing CreateTablet for tablet bf84b37806e44876838f1e2089610a9c (DEFAULT_TABLE table=test-table [id=52b9789a648d4d8585a6104e19b6c2a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:01.085300 24447 tablet_service.cc:1467] Processing CreateTablet for tablet bf84b37806e44876838f1e2089610a9c (DEFAULT_TABLE table=test-table [id=52b9789a648d4d8585a6104e19b6c2a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:01.085321 24728 tablet_service.cc:1467] Processing CreateTablet for tablet bf84b37806e44876838f1e2089610a9c (DEFAULT_TABLE table=test-table [id=52b9789a648d4d8585a6104e19b6c2a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:01.091228 24728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf84b37806e44876838f1e2089610a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:01.091228 24447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf84b37806e44876838f1e2089610a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:01.092650 24907 tablet_bootstrap.cc:492] T bf84b37806e44876838f1e2089610a9c P 86bd41d8b4634545a7c4e96e5e50f2d0: Bootstrap starting.
I20260928 14:38:01.093035 24907 tablet_bootstrap.cc:654] T bf84b37806e44876838f1e2089610a9c P 86bd41d8b4634545a7c4e96e5e50f2d0: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:01.093274 24896 ts_tablet_manager.cc:1918] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:38:01.093330 24896 log.cc:1196] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-0/wal/wals/191daa038cc24e74a9b2fb562471fbd8
I20260928 14:38:01.093320 24911 tablet_bootstrap.cc:492] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Bootstrap starting.
I20260928 14:38:01.093464 24897 ts_tablet_manager.cc:1918] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:38:01.093514 24897 log.cc:1196] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/191daa038cc24e74a9b2fb562471fbd8
I20260928 14:38:01.093570 24896 ts_tablet_manager.cc:1939] T 191daa038cc24e74a9b2fb562471fbd8 P 86bd41d8b4634545a7c4e96e5e50f2d0: Deleting consensus metadata
I20260928 14:38:01.093653 24911 tablet_bootstrap.cc:654] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:01.093725 24897 ts_tablet_manager.cc:1939] T 191daa038cc24e74a9b2fb562471fbd8 P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Deleting consensus metadata
I20260928 14:38:01.094245 24911 tablet_bootstrap.cc:492] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a: No bootstrap required, opened a new log
I20260928 14:38:01.094273 24907 tablet_bootstrap.cc:492] T bf84b37806e44876838f1e2089610a9c P 86bd41d8b4634545a7c4e96e5e50f2d0: No bootstrap required, opened a new log
I20260928 14:38:01.094269 24588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf84b37806e44876838f1e2089610a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:01.094295 24911 ts_tablet_manager.cc:1397] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:01.094316 24907 ts_tablet_manager.cc:1397] T bf84b37806e44876838f1e2089610a9c P 86bd41d8b4634545a7c4e96e5e50f2d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:38:01.094427 24911 raft_consensus.cc:348] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:01.094507 24911 raft_consensus.cc:374] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:01.094465 24907 raft_consensus.cc:348] T bf84b37806e44876838f1e2089610a9c P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:01.094533 24907 raft_consensus.cc:374] T bf84b37806e44876838f1e2089610a9c P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:01.094532 24911 raft_consensus.cc:729] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa41d15a6fd41d08b67a7a6f9ecb38a, State: Initialized, Role: FOLLOWER
I20260928 14:38:01.094553 24907 raft_consensus.cc:729] T bf84b37806e44876838f1e2089610a9c P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86bd41d8b4634545a7c4e96e5e50f2d0, State: Initialized, Role: FOLLOWER
I20260928 14:38:01.094594 24907 consensus_queue.cc:260] T bf84b37806e44876838f1e2089610a9c P 86bd41d8b4634545a7c4e96e5e50f2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:01.094587 24911 consensus_queue.cc:260] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:01.094674 24907 ts_tablet_manager.cc:1428] T bf84b37806e44876838f1e2089610a9c P 86bd41d8b4634545a7c4e96e5e50f2d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:01.094689 24911 ts_tablet_manager.cc:1428] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 14:38:01.095619 24906 tablet_bootstrap.cc:492] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21: Bootstrap starting.
I20260928 14:38:01.096138 24895 ts_tablet_manager.cc:1918] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:38:01.096179 24895 log.cc:1196] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/191daa038cc24e74a9b2fb562471fbd8
I20260928 14:38:01.096256 24906 tablet_bootstrap.cc:654] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:01.096338 24895 ts_tablet_manager.cc:1939] T 191daa038cc24e74a9b2fb562471fbd8 P abadc8072e634fd98393c935e8ed6d21: Deleting consensus metadata
I20260928 14:38:01.096879 24906 tablet_bootstrap.cc:492] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21: No bootstrap required, opened a new log
I20260928 14:38:01.096959 24906 ts_tablet_manager.cc:1397] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:01.097111 24906 raft_consensus.cc:348] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:01.097193 24906 raft_consensus.cc:374] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:01.097218 24906 raft_consensus.cc:729] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abadc8072e634fd98393c935e8ed6d21, State: Initialized, Role: FOLLOWER
I20260928 14:38:01.097271 24906 consensus_queue.cc:260] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:01.097404 24906 ts_tablet_manager.cc:1428] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:01.141160 24296 catalog_manager.cc:4841] TS 5aa41d15a6fd41d08b67a7a6f9ecb38a (127.22.13.67:34417): tablet 191daa038cc24e74a9b2fb562471fbd8 (table test-table [id=920eb916c2d1444c96de6be569355d07]) successfully deleted
I20260928 14:38:01.141294 24296 catalog_manager.cc:4841] TS 86bd41d8b4634545a7c4e96e5e50f2d0 (127.22.13.65:38091): tablet 191daa038cc24e74a9b2fb562471fbd8 (table test-table [id=920eb916c2d1444c96de6be569355d07]) successfully deleted
I20260928 14:38:01.174072 24297 catalog_manager.cc:4841] TS abadc8072e634fd98393c935e8ed6d21 (127.22.13.66:38305): tablet 191daa038cc24e74a9b2fb562471fbd8 (table test-table [id=920eb916c2d1444c96de6be569355d07]) successfully deleted
I20260928 14:38:01.199577 24910 raft_consensus.cc:482] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:38:01.199730 24910 raft_consensus.cc:504] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:01.200102 24910 leader_election.cc:288] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86bd41d8b4634545a7c4e96e5e50f2d0 (127.22.13.65:38091), 5aa41d15a6fd41d08b67a7a6f9ecb38a (127.22.13.67:34417)
I20260928 14:38:01.203231 24915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:01.200530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:33191 (local address 127.22.13.65:38091)
0928 14:38:01.200581 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:01.200583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:01.200598 (+ 15us) server_negotiation.cc:407] Connection header received
0928 14:38:01.200710 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:01.200711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:01.200724 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:01.200797 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:01.200974 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.201466 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.202499 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.202864 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.202958 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:01.202969 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:01.203116 (+ 147us) server_negotiation.cc:299] Negotiation successful
0928 14:38:01.203156 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260928 14:38:01.203258 24922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:01.200372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38091 (local address 127.22.13.66:33191)
0928 14:38:01.200526 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:01.200540 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:01.200618 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:01.200802 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:01.200803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:01.200805 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:01.200889 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:01.200891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.201558 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.201559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:01.202383 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:01.202385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.202956 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.202957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:01.203032 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:01.203036 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:38:01.203108 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 14:38:01.203151 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":70,"threads_started":1}
I20260928 14:38:01.203500 24923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:01.200578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:34417 (local address 127.22.13.66:52933)
0928 14:38:01.200734 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:01.200745 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:01.200811 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:01.200989 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:01.200990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:01.200992 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:01.201069 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:01.201071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.201781 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.201782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:01.202599 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:01.202601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.203178 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.203179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:01.203263 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:01.203272 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 14:38:01.203339 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 14:38:01.203389 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
I20260928 14:38:01.203472 24468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf84b37806e44876838f1e2089610a9c" candidate_uuid: "abadc8072e634fd98393c935e8ed6d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" is_pre_election: true
I20260928 14:38:01.203560 24468 raft_consensus.cc:2454] T bf84b37806e44876838f1e2089610a9c P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abadc8072e634fd98393c935e8ed6d21 in term 0.
I20260928 14:38:01.203636 24924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:01.200638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:52933 (local address 127.22.13.67:34417)
0928 14:38:01.200737 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:01.200739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:01.200794 (+ 55us) server_negotiation.cc:407] Connection header received
0928 14:38:01.200894 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:01.200895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:01.200904 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:01.200987 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:01.201152 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.201685 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.202696 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.203084 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.203170 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:01.203178 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:01.203444 (+ 266us) server_negotiation.cc:299] Negotiation successful
0928 14:38:01.203501 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":44,"threads_started":1}
I20260928 14:38:01.203744 24540 leader_election.cc:302] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86bd41d8b4634545a7c4e96e5e50f2d0, abadc8072e634fd98393c935e8ed6d21; no voters:
I20260928 14:38:01.203800 24748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf84b37806e44876838f1e2089610a9c" candidate_uuid: "abadc8072e634fd98393c935e8ed6d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" is_pre_election: true
I20260928 14:38:01.203871 24748 raft_consensus.cc:2454] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abadc8072e634fd98393c935e8ed6d21 in term 0.
I20260928 14:38:01.203907 24910 raft_consensus.cc:2781] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:38:01.203945 24910 raft_consensus.cc:482] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:38:01.203965 24910 raft_consensus.cc:3037] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:01.289098 24910 raft_consensus.cc:504] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:01.289327 24910 leader_election.cc:288] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [CANDIDATE]: Term 1 election: Requested vote from peers 86bd41d8b4634545a7c4e96e5e50f2d0 (127.22.13.65:38091), 5aa41d15a6fd41d08b67a7a6f9ecb38a (127.22.13.67:34417)
I20260928 14:38:01.289601 24468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf84b37806e44876838f1e2089610a9c" candidate_uuid: "abadc8072e634fd98393c935e8ed6d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0"
I20260928 14:38:01.289683 24748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf84b37806e44876838f1e2089610a9c" candidate_uuid: "abadc8072e634fd98393c935e8ed6d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a"
I20260928 14:38:01.289744 24468 raft_consensus.cc:3037] T bf84b37806e44876838f1e2089610a9c P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:01.289757 24748 raft_consensus.cc:3037] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:01.290584 24468 raft_consensus.cc:2454] T bf84b37806e44876838f1e2089610a9c P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abadc8072e634fd98393c935e8ed6d21 in term 1.
I20260928 14:38:01.290584 24748 raft_consensus.cc:2454] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abadc8072e634fd98393c935e8ed6d21 in term 1.
I20260928 14:38:01.290781 24540 leader_election.cc:302] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa41d15a6fd41d08b67a7a6f9ecb38a, abadc8072e634fd98393c935e8ed6d21; no voters:
I20260928 14:38:01.290887 24910 raft_consensus.cc:2781] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:01.291057 24910 raft_consensus.cc:686] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [term 1 LEADER]: Becoming Leader. State: Replica: abadc8072e634fd98393c935e8ed6d21, State: Running, Role: LEADER
I20260928 14:38:01.291146 24910 consensus_queue.cc:237] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } }
I20260928 14:38:01.291949 24309 catalog_manager.cc:5495] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 reported cstate change: term changed from 0 to 1, leader changed from <none> to abadc8072e634fd98393c935e8ed6d21 (127.22.13.66). New cstate: current_term: 1 leader_uuid: "abadc8072e634fd98393c935e8ed6d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abadc8072e634fd98393c935e8ed6d21" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 } health_report { overall_health: UNKNOWN } } }
I20260928 14:38:01.321375 24927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:01.318747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41316 (local address 127.22.13.66:38305)
0928 14:38:01.318940 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:01.318942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:01.318951 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:38:01.318997 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:01.318998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:01.319019 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:01.319094 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:38:01.319198 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.319794 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.320620 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.320818 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.320879 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:01.320999 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:38:01.321113 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:38:01.321166 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:01.321225 (+ 59us) server_negotiation.cc:299] Negotiation successful
0928 14:38:01.321241 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":138,"threads_started":1}
I20260928 14:38:01.326442 24915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:01.324103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:53077 (local address 127.22.13.65:38091)
0928 14:38:01.324154 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:01.324156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:01.324225 (+ 69us) server_negotiation.cc:407] Connection header received
0928 14:38:01.324279 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:01.324280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:01.324290 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:01.324340 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:38:01.324457 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.324921 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.325753 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.325978 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.326029 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:01.326198 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:38:01.326224 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:38:01.326263 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:01.326360 (+ 97us) server_negotiation.cc:299] Negotiation successful
0928 14:38:01.326380 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:38:01.326499 24930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:01.324015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38091 (local address 127.22.13.66:53077)
0928 14:38:01.324167 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:01.324179 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:01.324238 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:01.324351 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:01.324352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:01.324354 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:38:01.324409 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:01.324411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.325002 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.325003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:01.325634 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:01.325636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.326057 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.326058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:01.326108 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:01.326123 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:38:01.326304 (+ 181us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:38:01.326306 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:38:01.326360 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 14:38:01.326407 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260928 14:38:01.328621 24748 raft_consensus.cc:1261] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Refusing update from remote peer abadc8072e634fd98393c935e8ed6d21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:01.328701 24468 raft_consensus.cc:1261] T bf84b37806e44876838f1e2089610a9c P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 FOLLOWER]: Refusing update from remote peer abadc8072e634fd98393c935e8ed6d21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:01.328943 24910 consensus_queue.cc:1035] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aa41d15a6fd41d08b67a7a6f9ecb38a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:01.329083 24925 consensus_queue.cc:1035] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86bd41d8b4634545a7c4e96e5e50f2d0" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:01.337451 24927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:01.335015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:59645 (local address 127.22.13.66:38305)
0928 14:38:01.335078 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:01.335080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:01.335152 (+ 72us) server_negotiation.cc:407] Connection header received
0928 14:38:01.335222 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:01.335223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:01.335253 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:01.335308 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:38:01.335418 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.335983 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.336746 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.336950 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.337035 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:01.337190 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:38:01.337226 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:38:01.337290 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:01.337352 (+ 62us) server_negotiation.cc:299] Negotiation successful
0928 14:38:01.337374 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:38:01.337487 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:01.334932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:38305 (local address 127.22.13.65:59645)
0928 14:38:01.335100 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:01.335110 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:01.335170 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:01.335312 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:01.335313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:01.335314 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:38:01.335370 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:01.335371 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.336078 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.336079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:01.336655 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:01.336657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:01.337037 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:01.337037 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:01.337095 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:01.337102 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:38:01.337292 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:38:01.337294 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:38:01.337347 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 14:38:01.337397 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
W20260928 14:38:01.338037 24878 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260928 14:38:02.861636 24309 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:02.861738 24309 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:02.862661 24309 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a3dd7775ab7468d807ddd3c8828a3a7: Sending DeleteTablet for 3 replicas of tablet bf84b37806e44876838f1e2089610a9c
I20260928 14:38:02.862972 24588 tablet_service.cc:1514] Processing DeleteTablet for tablet bf84b37806e44876838f1e2089610a9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:02 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:41300
I20260928 14:38:02.863019 24448 tablet_service.cc:1514] Processing DeleteTablet for tablet bf84b37806e44876838f1e2089610a9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:02 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:45504
I20260928 14:38:02.863107 24728 tablet_service.cc:1514] Processing DeleteTablet for tablet bf84b37806e44876838f1e2089610a9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:02 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:34874
I20260928 14:38:02.863214 24959 tablet_replica.cc:330] T bf84b37806e44876838f1e2089610a9c P 86bd41d8b4634545a7c4e96e5e50f2d0: stopping tablet replica
I20260928 14:38:02.863276 24960 tablet_replica.cc:330] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a: stopping tablet replica
I20260928 14:38:02.863298 24959 raft_consensus.cc:2229] T bf84b37806e44876838f1e2089610a9c P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:02.863312 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 24383
I20260928 14:38:02.863350 24959 raft_consensus.cc:2258] T bf84b37806e44876838f1e2089610a9c P 86bd41d8b4634545a7c4e96e5e50f2d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:02.863380 24960 raft_consensus.cc:2229] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:02.864059 24960 raft_consensus.cc:2258] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:02.864338 24960 ts_tablet_manager.cc:1905] T bf84b37806e44876838f1e2089610a9c P 5aa41d15a6fd41d08b67a7a6f9ecb38a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:02.870848 24958 tablet_replica.cc:330] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21: stopping tablet replica
I20260928 14:38:02.870949 24958 raft_consensus.cc:2229] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:38:02.871084 24958 raft_consensus.cc:2258] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260928 14:38:02.871336 24296 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:38091 (error 108)
I20260928 14:38:02.871488 24958 ts_tablet_manager.cc:1905] T bf84b37806e44876838f1e2089610a9c P abadc8072e634fd98393c935e8ed6d21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:02.871650 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 24523
I20260928 14:38:02.872447 24962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:02.871809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38091 (local address 127.0.0.1:45540)
0928 14:38:02.872335 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:02.872395 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:38091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":60,"threads_started":1}
W20260928 14:38:02.872589 24296 catalog_manager.cc:4584] TS 86bd41d8b4634545a7c4e96e5e50f2d0 (127.22.13.65:38091): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf84b37806e44876838f1e2089610a9c: Network error: Client connection negotiation failed: client connection to 127.22.13.65:38091: connect: Connection refused (error 111)
I20260928 14:38:02.878206 24962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:02.878080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:38305 (local address 127.0.0.1:41318)
0928 14:38:02.878138 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:02.878167 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.66:38305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260928 14:38:02.878343 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 24663
I20260928 14:38:02.915602 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 24279
2026-09-28T14:38:02Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6003 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:38:04 dist-test-slave-wwwb krb5kdc[24965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:38:04 dist-test-slave-wwwb krb5kdc[24965](info): set up 2 sockets
Sep 28 14:38:04 dist-test-slave-wwwb krb5kdc[24965](info): commencing operation
krb5kdc: starting...
W20260928 14:38:05.634976 22581 mini_kdc.cc:121] Time spent starting KDC: real 2.697s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:38:05 dist-test-slave-wwwb krb5kdc[24965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:38:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:38:05.764669 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:33131
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:33131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:05.849457 24978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:05.849737 24978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:05.849785 24978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:05.851066 24978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:05.851157 24978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:05.851179 24978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:05.851199 24978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:05.851229 24978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:05.852845 24978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:33131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:33131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:05.853148 24978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:05.853389 24978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:38:05.858790 24978 server_base.cc:1034] running on GCE node
W20260928 14:38:05.858777 24983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:05.858750 24986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:05.858733 24984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:05.859308 24978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:05.859652 24978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:05.860841 24978 hybrid_clock.cc:648] HybridClock initialized: now 1790606285860822 us; error 57 us; skew 500 ppm
Sep 28 14:38:05 dist-test-slave-wwwb krb5kdc[24965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606285, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:05.870388 24978 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:38:05.870814 24978 webserver.cc:460] Webserver started at http://127.22.13.126:44389/ using document root <none> and password file <none>
I20260928 14:38:05.871011 24978 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:05.871052 24978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:05.871142 24978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:05.872177 24978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "e09c03159a394967ae10fcf2b114bfb8"
format_stamp: "Formatted at 2026-09-28 14:38:05 on dist-test-slave-wwwb"
server_key: "6cacab3561065bb709661da636062f6f"
server_key_iv: "62763ca345ef3eaec14bd941cf8fe847"
server_key_version: "encryptionkey@0"
I20260928 14:38:05.872491 24978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "e09c03159a394967ae10fcf2b114bfb8"
format_stamp: "Formatted at 2026-09-28 14:38:05 on dist-test-slave-wwwb"
server_key: "6cacab3561065bb709661da636062f6f"
server_key_iv: "62763ca345ef3eaec14bd941cf8fe847"
server_key_version: "encryptionkey@0"
I20260928 14:38:05.873804 24978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 14:38:05.874599 24993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:05.874816 24978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:05.874927 24978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "e09c03159a394967ae10fcf2b114bfb8"
format_stamp: "Formatted at 2026-09-28 14:38:05 on dist-test-slave-wwwb"
server_key: "6cacab3561065bb709661da636062f6f"
server_key_iv: "62763ca345ef3eaec14bd941cf8fe847"
server_key_version: "encryptionkey@0"
I20260928 14:38:05.874991 24978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:05.891666 24978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:05.894145 24978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:05.894269 24978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:05.898061 24978 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:33131
I20260928 14:38:05.898137 25045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:33131 every 8 connection(s)
I20260928 14:38:05.898490 24978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:05.899451 25046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:05.900432 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 24978
I20260928 14:38:05.900537 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:05.900718 22581 external_mini_cluster.cc:1428] Setting key 4686811f4b2c719d234c378c1c2c0545
I20260928 14:38:05.903021 25046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8: Bootstrap starting.
I20260928 14:38:05.903668 25046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:05.903980 25046 log.cc:824] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:05.904803 25046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8: No bootstrap required, opened a new log
I20260928 14:38:05.907498 25046 raft_consensus.cc:348] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09c03159a394967ae10fcf2b114bfb8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 33131 } }
I20260928 14:38:05.907651 25046 raft_consensus.cc:374] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:05.907673 25046 raft_consensus.cc:729] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e09c03159a394967ae10fcf2b114bfb8, State: Initialized, Role: FOLLOWER
I20260928 14:38:05.907812 25046 consensus_queue.cc:260] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09c03159a394967ae10fcf2b114bfb8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 33131 } }
I20260928 14:38:05.907900 25046 raft_consensus.cc:388] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Sep 28 14:38:05 dist-test-slave-wwwb krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:05.907925 25046 raft_consensus.cc:482] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:05.907987 25046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:05.908777 25046 raft_consensus.cc:504] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09c03159a394967ae10fcf2b114bfb8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 33131 } }
I20260928 14:38:05.908913 25046 leader_election.cc:302] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e09c03159a394967ae10fcf2b114bfb8; no voters:
I20260928 14:38:05.909125 25046 leader_election.cc:288] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:05.909163 25051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:05.909274 25051 raft_consensus.cc:686] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [term 1 LEADER]: Becoming Leader. State: Replica: e09c03159a394967ae10fcf2b114bfb8, State: Running, Role: LEADER
I20260928 14:38:05.909411 25046 sys_catalog.cc:564] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:05.909397 25051 consensus_queue.cc:237] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09c03159a394967ae10fcf2b114bfb8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 33131 } }
I20260928 14:38:05.910038 25052 sys_catalog.cc:455] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e09c03159a394967ae10fcf2b114bfb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09c03159a394967ae10fcf2b114bfb8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 33131 } } }
I20260928 14:38:05.910193 25052 sys_catalog.cc:458] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:05.910626 25061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:05.910840 25053 sys_catalog.cc:455] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e09c03159a394967ae10fcf2b114bfb8. Latest consensus state: current_term: 1 leader_uuid: "e09c03159a394967ae10fcf2b114bfb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09c03159a394967ae10fcf2b114bfb8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 33131 } } }
I20260928 14:38:05.911072 25053 sys_catalog.cc:458] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8 [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:05.911309 25061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:05.912292 25049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:05.901309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41018 (local address 127.22.13.126:33131)
0928 14:38:05.901481 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:05.901492 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:05.901527 (+ 35us) server_negotiation.cc:407] Connection header received
0928 14:38:05.902016 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:05.902025 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:05.902087 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:05.902212 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:05.903029 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:05.904082 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:05.905060 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:05.905468 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:05.905556 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:05.908410 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:05.908414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:05.908415 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:05.908428 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:05.910515 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:05.910987 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:05.910991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:05.910991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:05.911105 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:05.911340 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:05.911341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:05.911341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:05.911665 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:05.911825 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:05.911960 (+ 135us) server_negotiation.cc:299] Negotiation successful
0928 14:38:05.912104 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
I20260928 14:38:05.913223 25061 catalog_manager.cc:1347] Generated new cluster ID: f1dee804490e49c3a9e2d0caf27f9d91
I20260928 14:38:05.913262 25061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:38:05.921475 25061 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:05.922135 25061 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:05.925024 25061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e09c03159a394967ae10fcf2b114bfb8: Generated new TSK 0
I20260928 14:38:05.925235 25061 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:38:05.952735 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:33131
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:06.054327 25072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:06.054569 25072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:06.054608 25072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:06.055939 25072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:06.056013 25072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:06.056061 25072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:38:06.057577 25072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:33131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:06.057865 25072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:06.058137 25072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:06.064093 25077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:06.064064 25080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:06.064067 25078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:06.064483 25072 server_base.cc:1034] running on GCE node
I20260928 14:38:06.064720 25072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:06.065007 25072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:06.066175 25072 hybrid_clock.cc:648] HybridClock initialized: now 1790606286066162 us; error 34 us; skew 500 ppm
Sep 28 14:38:06 dist-test-slave-wwwb krb5kdc[24965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606286, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:06.069063 25072 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:38:06.069626 25072 webserver.cc:460] Webserver started at http://127.22.13.65:34613/ using document root <none> and password file <none>
I20260928 14:38:06.069895 25072 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:06.069988 25072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:06.070148 25072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:06.071389 25072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "083c55b6817b44cc8e32c6e3d8175e9d"
format_stamp: "Formatted at 2026-09-28 14:38:06 on dist-test-slave-wwwb"
server_key: "5307d731b6fa6f261fed1d4c87058fd5"
server_key_iv: "efebdbb8186280d66674ffd248dd26a9"
server_key_version: "encryptionkey@0"
I20260928 14:38:06.071754 25072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "083c55b6817b44cc8e32c6e3d8175e9d"
format_stamp: "Formatted at 2026-09-28 14:38:06 on dist-test-slave-wwwb"
server_key: "5307d731b6fa6f261fed1d4c87058fd5"
server_key_iv: "efebdbb8186280d66674ffd248dd26a9"
server_key_version: "encryptionkey@0"
I20260928 14:38:06.073134 25072 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:06.074002 25087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:06.074203 25072 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:06.074307 25072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "083c55b6817b44cc8e32c6e3d8175e9d"
format_stamp: "Formatted at 2026-09-28 14:38:06 on dist-test-slave-wwwb"
server_key: "5307d731b6fa6f261fed1d4c87058fd5"
server_key_iv: "efebdbb8186280d66674ffd248dd26a9"
server_key_version: "encryptionkey@0"
I20260928 14:38:06.074402 25072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:06.084390 25072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:06.086411 25072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:06.086580 25072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:06.086820 25072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:06.087175 25072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:06.087231 25072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:06.087277 25072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:06.087301 25072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:06.093319 25072 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:43659
I20260928 14:38:06.093384 25200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:43659 every 8 connection(s)
I20260928 14:38:06.093714 25072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:38:06.098531 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 25072
I20260928 14:38:06.098642 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:38:06.098819 22581 external_mini_cluster.cc:1428] Setting key 792dfd1b9cd0450c35c73766ad2fa5ff
Sep 28 14:38:06 dist-test-slave-wwwb krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606286, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:06.103802 25049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:06.094437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:55219 (local address 127.22.13.126:33131)
0928 14:38:06.094543 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:06.094545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:06.094730 (+ 185us) server_negotiation.cc:407] Connection header received
0928 14:38:06.095203 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:06.095205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:06.095215 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:06.095271 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:06.096393 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:06.096899 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:06.097890 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:06.098187 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:06.098240 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:06.100739 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:06.100740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:06.100741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:06.100749 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:06.102213 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:06.102728 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:06.102730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:06.102730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:06.102794 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:06.103085 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:06.103086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:06.103086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:06.103195 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:06.103248 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:06.103591 (+ 343us) server_negotiation.cc:299] Negotiation successful
0928 14:38:06.103672 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 14:38:06.104097 25203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:06.094408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:33131 (local address 127.22.13.65:55219)
0928 14:38:06.094601 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:06.094639 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:06.095105 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:06.095484 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:06.095488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:06.096077 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:06.096290 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:06.096303 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:06.096982 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:06.096983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:06.097808 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:06.097810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:06.098261 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:06.098262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:06.098441 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:06.099085 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:06.099094 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:06.100626 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:06.102299 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:06.102307 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:06.102310 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:06.102642 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:06.102868 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:06.102869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:06.102869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:06.103021 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:06.103260 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:06.103267 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:06.103448 (+ 181us) client_negotiation.cc:769] Sending connection context
0928 14:38:06.103573 (+ 125us) client_negotiation.cc:240] Negotiation successful
0928 14:38:06.103780 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":118,"threads_started":1}
I20260928 14:38:06.104687 25201 heartbeater.cc:347] Connected to a master server at 127.22.13.126:33131
I20260928 14:38:06.104782 25201 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:06.104981 25201 heartbeater.cc:511] Master 127.22.13.126:33131 requested a full tablet report, sending...
I20260928 14:38:06.105583 25010 ts_manager.cc:194] Registered new tserver with Master: 083c55b6817b44cc8e32c6e3d8175e9d (127.22.13.65:43659)
I20260928 14:38:06.106292 25010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:55219
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:38:06.124320 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:33131
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:06.206537 25206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:06.206786 25206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:06.206827 25206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:06.208086 25206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:06.208171 25206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:06.208220 25206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:38:06.209805 25206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:33131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:06.210114 25206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:06.210355 25206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:06.215281 25211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:06.215284 25214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:06.215289 25212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:06.215538 25206 server_base.cc:1034] running on GCE node
I20260928 14:38:06.215782 25206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:06.216061 25206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:06.217226 25206 hybrid_clock.cc:648] HybridClock initialized: now 1790606286217212 us; error 28 us; skew 500 ppm
Sep 28 14:38:06 dist-test-slave-wwwb krb5kdc[24965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606286, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:06.219695 25206 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:38:06.220124 25206 webserver.cc:460] Webserver started at http://127.22.13.66:39445/ using document root <none> and password file <none>
I20260928 14:38:06.220364 25206 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:06.220432 25206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:06.220579 25206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:06.221612 25206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "7f16a4a5d6834af38589b5cddc863347"
format_stamp: "Formatted at 2026-09-28 14:38:06 on dist-test-slave-wwwb"
server_key: "dfb125f38848021a5de9fe50496d733e"
server_key_iv: "dc4cdb1d8b800139391da8122775a841"
server_key_version: "encryptionkey@0"
I20260928 14:38:06.222028 25206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "7f16a4a5d6834af38589b5cddc863347"
format_stamp: "Formatted at 2026-09-28 14:38:06 on dist-test-slave-wwwb"
server_key: "dfb125f38848021a5de9fe50496d733e"
server_key_iv: "dc4cdb1d8b800139391da8122775a841"
server_key_version: "encryptionkey@0"
I20260928 14:38:06.223385 25206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:06.224224 25221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:06.224401 25206 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:06.224498 25206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "7f16a4a5d6834af38589b5cddc863347"
format_stamp: "Formatted at 2026-09-28 14:38:06 on dist-test-slave-wwwb"
server_key: "dfb125f38848021a5de9fe50496d733e"
server_key_iv: "dc4cdb1d8b800139391da8122775a841"
server_key_version: "encryptionkey@0"
I20260928 14:38:06.224586 25206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:06.236934 25206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:06.239611 25206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:06.239794 25206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:06.240042 25206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:06.240360 25206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:06.240413 25206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:06.240447 25206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:06.240494 25206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:06.246321 25206 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:43399
I20260928 14:38:06.246372 25334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:43399 every 8 connection(s)
I20260928 14:38:06.246706 25206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:38:06.249609 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 25206
I20260928 14:38:06.249696 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:38:06.249905 22581 external_mini_cluster.cc:1428] Setting key f59b0fd9a262283077c3d47a63475914
Sep 28 14:38:06 dist-test-slave-wwwb krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606286, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:06.256975 25049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:06.247413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:46735 (local address 127.22.13.126:33131)
0928 14:38:06.247488 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:06.247490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:06.247700 (+ 210us) server_negotiation.cc:407] Connection header received
0928 14:38:06.248216 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:06.248217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:06.248227 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:06.248294 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:06.249269 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:06.249832 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:06.251329 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:06.251520 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:06.251576 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:06.253928 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:06.253967 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:06.253967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:06.253976 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:06.255395 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:06.255951 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:06.255952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:06.255952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:06.256005 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:06.256310 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:06.256310 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:06.256310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:06.256417 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:06.256467 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:06.256836 (+ 369us) server_negotiation.cc:299] Negotiation successful
0928 14:38:06.256907 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 14:38:06.257278 25337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:06.247395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:33131 (local address 127.22.13.66:46735)
0928 14:38:06.247552 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:06.247597 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:06.248122 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:06.248383 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:06.248387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:06.248969 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:06.249177 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:06.249181 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:06.249912 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:06.249914 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:06.251235 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:06.251238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:06.251607 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:06.251608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:06.251775 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:06.252301 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:06.252311 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:06.253808 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:06.255484 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:06.255492 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:06.255495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:06.255869 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:06.256074 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:06.256075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:06.256075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:06.256238 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:06.256485 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:06.256492 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:06.256686 (+ 194us) client_negotiation.cc:769] Sending connection context
0928 14:38:06.256819 (+ 133us) client_negotiation.cc:240] Negotiation successful
0928 14:38:06.257029 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260928 14:38:06.257874 25335 heartbeater.cc:347] Connected to a master server at 127.22.13.126:33131
I20260928 14:38:06.258016 25335 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:06.258231 25335 heartbeater.cc:511] Master 127.22.13.126:33131 requested a full tablet report, sending...
I20260928 14:38:06.258649 25010 ts_manager.cc:194] Registered new tserver with Master: 7f16a4a5d6834af38589b5cddc863347 (127.22.13.66:43399)
I20260928 14:38:06.259044 25010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:46735
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:38:06.276050 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:33131
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:35959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:06.358129 25340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:06.358361 25340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:06.358386 25340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:06.359543 25340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:06.359583 25340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:06.359609 25340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:38:06.361406 25340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:35959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:33131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:06.361687 25340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:06.361948 25340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:06.367027 25345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:06.367053 25346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:06.367123 25340 server_base.cc:1034] running on GCE node
W20260928 14:38:06.367031 25348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:06.367502 25340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:06.367787 25340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:06.368959 25340 hybrid_clock.cc:648] HybridClock initialized: now 1790606286368950 us; error 34 us; skew 500 ppm
Sep 28 14:38:06 dist-test-slave-wwwb krb5kdc[24965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606286, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:06.371533 25340 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:38:06.372031 25340 webserver.cc:460] Webserver started at http://127.22.13.67:46853/ using document root <none> and password file <none>
I20260928 14:38:06.372262 25340 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:06.372332 25340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:06.372504 25340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:06.373608 25340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "dc3014194b7d4b77b30a53772e65c5e7"
format_stamp: "Formatted at 2026-09-28 14:38:06 on dist-test-slave-wwwb"
server_key: "0c2b5bfea26e2b62e37198c576084f73"
server_key_iv: "b070d6c77fad395601f507bbd166514f"
server_key_version: "encryptionkey@0"
I20260928 14:38:06.374071 25340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "dc3014194b7d4b77b30a53772e65c5e7"
format_stamp: "Formatted at 2026-09-28 14:38:06 on dist-test-slave-wwwb"
server_key: "0c2b5bfea26e2b62e37198c576084f73"
server_key_iv: "b070d6c77fad395601f507bbd166514f"
server_key_version: "encryptionkey@0"
I20260928 14:38:06.375451 25340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:06.376171 25355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:06.376372 25340 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:06.376461 25340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "dc3014194b7d4b77b30a53772e65c5e7"
format_stamp: "Formatted at 2026-09-28 14:38:06 on dist-test-slave-wwwb"
server_key: "0c2b5bfea26e2b62e37198c576084f73"
server_key_iv: "b070d6c77fad395601f507bbd166514f"
server_key_version: "encryptionkey@0"
I20260928 14:38:06.376546 25340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:06.383567 25340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:06.385377 25340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:06.385533 25340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:06.385756 25340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:06.386142 25340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:06.386196 25340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:06.386235 25340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:06.386263 25340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:06.392120 25340 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:39267
I20260928 14:38:06.392161 25468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:39267 every 8 connection(s)
I20260928 14:38:06.392515 25340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
Sep 28 14:38:06 dist-test-slave-wwwb krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606286, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:06.401322 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 25340
I20260928 14:38:06.401454 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:38:06.401647 22581 external_mini_cluster.cc:1428] Setting key 260171d488440148c95bb2ef5c226559
I20260928 14:38:06.402148 25049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:06.393287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:36675 (local address 127.22.13.126:33131)
0928 14:38:06.393335 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:06.393337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:06.393537 (+ 200us) server_negotiation.cc:407] Connection header received
0928 14:38:06.394151 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:06.394152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:06.394165 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:06.394227 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:06.395245 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:06.395764 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:06.396796 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:06.397002 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:06.397079 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:06.399328 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:06.399329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:06.399330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:06.399339 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:06.400567 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:06.401004 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:06.401006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:06.401006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:06.401073 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:06.401325 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:06.401326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:06.401326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:06.401483 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:06.401550 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:06.401927 (+ 377us) server_negotiation.cc:299] Negotiation successful
0928 14:38:06.402041 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:38:06.402384 25471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:06.393252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:33131 (local address 127.22.13.67:36675)
0928 14:38:06.393409 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:06.393446 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:06.394047 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:06.394303 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:06.394307 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:06.394925 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:06.395128 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:06.395134 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:06.395868 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:06.395869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:06.396680 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:06.396683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:06.397086 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:06.397087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:06.397220 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:06.397771 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:06.397780 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:06.399220 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:06.400655 (+ 1435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:06.400662 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:06.400665 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:06.400903 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:06.401144 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:06.401145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:06.401145 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:06.401256 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:06.401558 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:06.401564 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:06.401737 (+ 173us) client_negotiation.cc:769] Sending connection context
0928 14:38:06.401865 (+ 128us) client_negotiation.cc:240] Negotiation successful
0928 14:38:06.402082 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":96,"threads_started":1}
I20260928 14:38:06.402988 25469 heartbeater.cc:347] Connected to a master server at 127.22.13.126:33131
I20260928 14:38:06.403100 25469 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:06.403282 25469 heartbeater.cc:511] Master 127.22.13.126:33131 requested a full tablet report, sending...
I20260928 14:38:06.403635 25010 ts_manager.cc:194] Registered new tserver with Master: dc3014194b7d4b77b30a53772e65c5e7 (127.22.13.67:39267)
I20260928 14:38:06.404017 25010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:36675
I20260928 14:38:06.413463 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 14:38:06.424186 25049 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:06.423233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41034 (local address 127.22.13.126:33131)
0928 14:38:06.423327 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:06.423329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:06.423387 (+ 58us) server_negotiation.cc:407] Connection header received
0928 14:38:06.423448 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:06.423449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:06.423459 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:06.423511 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:06.424107 (+ 596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41034: BlockingRecv error: recv got EOF from 127.0.0.1:41034 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 14:38:06.425280 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 25072
I20260928 14:38:06.430614 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 25206
I20260928 14:38:06.435706 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 25340
I20260928 14:38:06.439934 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 24978
2026-09-28T14:38:06Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3523 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:38:06 dist-test-slave-wwwb krb5kdc[25481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:38:06 dist-test-slave-wwwb krb5kdc[25481](info): set up 2 sockets
Sep 28 14:38:06 dist-test-slave-wwwb krb5kdc[25481](info): commencing operation
krb5kdc: starting...
W20260928 14:38:07.458027 22581 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:38:07 dist-test-slave-wwwb krb5kdc[25481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:38:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:38:07.578433 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:43235
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:43235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:07.659927 25494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:07.660159 25494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:07.660183 25494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:07.661448 25494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:07.661499 25494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:07.661513 25494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:07.661527 25494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:07.661540 25494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:07.663142 25494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:43235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:43235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:07.663440 25494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:07.663686 25494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:07.668542 25502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:07.668545 25500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:07.668524 25499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:07.668831 25494 server_base.cc:1034] running on GCE node
I20260928 14:38:07.669060 25494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:07.669304 25494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:07.670493 25494 hybrid_clock.cc:648] HybridClock initialized: now 1790606287670479 us; error 26 us; skew 500 ppm
Sep 28 14:38:07 dist-test-slave-wwwb krb5kdc[25481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606287, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:07.673147 25494 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:38:07.673671 25494 webserver.cc:460] Webserver started at http://127.22.13.126:41057/ using document root <none> and password file <none>
I20260928 14:38:07.673902 25494 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:07.673992 25494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:07.674151 25494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:07.675244 25494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "7abfc7b96c7d4b468c72b3ff33c37c23"
format_stamp: "Formatted at 2026-09-28 14:38:07 on dist-test-slave-wwwb"
server_key: "eaa964af00a4f8d9379d119856c35ea7"
server_key_iv: "774db5b34f17511489fc1ccb72090de3"
server_key_version: "encryptionkey@0"
I20260928 14:38:07.675633 25494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "7abfc7b96c7d4b468c72b3ff33c37c23"
format_stamp: "Formatted at 2026-09-28 14:38:07 on dist-test-slave-wwwb"
server_key: "eaa964af00a4f8d9379d119856c35ea7"
server_key_iv: "774db5b34f17511489fc1ccb72090de3"
server_key_version: "encryptionkey@0"
I20260928 14:38:07.677031 25494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:07.677765 25509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:07.677965 25494 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:07.678073 25494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "7abfc7b96c7d4b468c72b3ff33c37c23"
format_stamp: "Formatted at 2026-09-28 14:38:07 on dist-test-slave-wwwb"
server_key: "eaa964af00a4f8d9379d119856c35ea7"
server_key_iv: "774db5b34f17511489fc1ccb72090de3"
server_key_version: "encryptionkey@0"
I20260928 14:38:07.678160 25494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:07.691856 25494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:07.694336 25494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:07.694507 25494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:07.698547 25494 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:43235
I20260928 14:38:07.698585 25561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:43235 every 8 connection(s)
I20260928 14:38:07.699101 25494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:07.699869 25562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:07.702245 25562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23: Bootstrap starting.
I20260928 14:38:07.702797 25562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:07.703068 25562 log.cc:824] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:07.703737 25562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23: No bootstrap required, opened a new log
I20260928 14:38:07.703941 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 25494
I20260928 14:38:07.704053 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:07.704233 22581 external_mini_cluster.cc:1428] Setting key c0834e852a8ed2f31db73bb27ce9748d
I20260928 14:38:07.706404 25562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abfc7b96c7d4b468c72b3ff33c37c23" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43235 } }
I20260928 14:38:07.706607 25562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:07.706652 25562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7abfc7b96c7d4b468c72b3ff33c37c23, State: Initialized, Role: FOLLOWER
I20260928 14:38:07.706812 25562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abfc7b96c7d4b468c72b3ff33c37c23" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43235 } }
I20260928 14:38:07.706928 25562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:07.707010 25562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:07.707078 25562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:07.707782 25562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abfc7b96c7d4b468c72b3ff33c37c23" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43235 } }
I20260928 14:38:07.707932 25562 leader_election.cc:302] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7abfc7b96c7d4b468c72b3ff33c37c23; no voters:
I20260928 14:38:07.708125 25562 leader_election.cc:288] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:07.708158 25567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:07.708278 25567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [term 1 LEADER]: Becoming Leader. State: Replica: 7abfc7b96c7d4b468c72b3ff33c37c23, State: Running, Role: LEADER
I20260928 14:38:07.708402 25562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:07.708431 25567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abfc7b96c7d4b468c72b3ff33c37c23" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43235 } }
I20260928 14:38:07.708884 25569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7abfc7b96c7d4b468c72b3ff33c37c23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abfc7b96c7d4b468c72b3ff33c37c23" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43235 } } }
I20260928 14:38:07.709014 25569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:07.708897 25568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7abfc7b96c7d4b468c72b3ff33c37c23. Latest consensus state: current_term: 1 leader_uuid: "7abfc7b96c7d4b468c72b3ff33c37c23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7abfc7b96c7d4b468c72b3ff33c37c23" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43235 } } }
I20260928 14:38:07.709153 25568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23 [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:07.710038 25575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:07.710602 25575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 28 14:38:07 dist-test-slave-wwwb krb5kdc[25481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:07.716666 25566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:07.704776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35288 (local address 127.22.13.126:43235)
0928 14:38:07.704962 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:07.704970 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:07.704995 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:38:07.706340 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:07.706348 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:07.706404 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:07.706531 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:07.707436 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:07.708590 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:07.709663 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:07.710351 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:07.710471 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:07.713111 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:07.713118 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:07.713119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:07.713134 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:07.715060 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:07.715467 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:07.715471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:07.715472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:07.715581 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:07.715806 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:07.715806 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:07.715807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:07.716067 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:07.716226 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:07.716343 (+ 117us) server_negotiation.cc:299] Negotiation successful
0928 14:38:07.716486 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":95,"threads_started":1}
I20260928 14:38:07.717700 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:38:07.718381 25575 catalog_manager.cc:1347] Generated new cluster ID: 17829c25368e4c10a21231142cd7d863
I20260928 14:38:07.718454 25575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:38:07.722250 25575 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:07.722908 25575 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:07.734985 25575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7abfc7b96c7d4b468c72b3ff33c37c23: Generated new TSK 0
I20260928 14:38:07.735212 25575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:38:07.736589 25566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:07.731917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35300 (local address 127.22.13.126:43235)
0928 14:38:07.731981 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:07.731983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:07.732093 (+ 110us) server_negotiation.cc:407] Connection header received
0928 14:38:07.732149 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:07.732151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:07.732162 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:07.732242 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:07.732718 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:07.733546 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:07.734914 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:07.735259 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:07.735331 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:07.736120 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:07.736122 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:07.736122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:07.736300 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:07.736318 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:07.736325 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:07.736374 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:07.736491 (+ 117us) server_negotiation.cc:299] Negotiation successful
0928 14:38:07.736535 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:38:07.738978 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 25494
2026-09-28T14:38:07Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1295 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:38:07 dist-test-slave-wwwb krb5kdc[25595](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:38:07 dist-test-slave-wwwb krb5kdc[25595](info): set up 2 sockets
Sep 28 14:38:07 dist-test-slave-wwwb krb5kdc[25595](info): commencing operation
krb5kdc: starting...
W20260928 14:38:08.758231 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:38:08 dist-test-slave-wwwb krb5kdc[25595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:38:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:38:08.876590 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:46623
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:46623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:08.958623 25608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:08.958849 25608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:08.958875 25608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:08.960037 25608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:08.960078 25608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:08.960093 25608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:08.960105 25608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:08.960117 25608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:08.961608 25608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:46623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:46623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:08.961822 25608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:08.962097 25608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:08.967108 25616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:08.967108 25614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:08.967123 25613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:08.967684 25608 server_base.cc:1034] running on GCE node
I20260928 14:38:08.967923 25608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:08.968236 25608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:08.969388 25608 hybrid_clock.cc:648] HybridClock initialized: now 1790606288969367 us; error 37 us; skew 500 ppm
Sep 28 14:38:08 dist-test-slave-wwwb krb5kdc[25595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606288, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:08.972066 25608 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:38:08.972580 25608 webserver.cc:460] Webserver started at http://127.22.13.126:38797/ using document root <none> and password file <none>
I20260928 14:38:08.972846 25608 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:08.972892 25608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:08.973069 25608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:08.974249 25608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "1e7be41b68634382813168be93ff292c"
format_stamp: "Formatted at 2026-09-28 14:38:08 on dist-test-slave-wwwb"
server_key: "c6c1ba65682e83c0e89d255618dbf510"
server_key_iv: "64b3a9c4c582b4d25d8079fc2db35a09"
server_key_version: "encryptionkey@0"
I20260928 14:38:08.974617 25608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "1e7be41b68634382813168be93ff292c"
format_stamp: "Formatted at 2026-09-28 14:38:08 on dist-test-slave-wwwb"
server_key: "c6c1ba65682e83c0e89d255618dbf510"
server_key_iv: "64b3a9c4c582b4d25d8079fc2db35a09"
server_key_version: "encryptionkey@0"
I20260928 14:38:08.976012 25608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:38:08.976778 25623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:08.976970 25608 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:08.977067 25608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "1e7be41b68634382813168be93ff292c"
format_stamp: "Formatted at 2026-09-28 14:38:08 on dist-test-slave-wwwb"
server_key: "c6c1ba65682e83c0e89d255618dbf510"
server_key_iv: "64b3a9c4c582b4d25d8079fc2db35a09"
server_key_version: "encryptionkey@0"
I20260928 14:38:08.977149 25608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:08.997781 25608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:08.999786 25608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:08.999941 25608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:09.004455 25608 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:46623
I20260928 14:38:09.004535 25675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:46623 every 8 connection(s)
I20260928 14:38:09.004879 25608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:09.006101 25676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:09.008590 25676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c: Bootstrap starting.
I20260928 14:38:09.009240 25676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:09.009558 25676 log.cc:824] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:09.010526 25676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c: No bootstrap required, opened a new log
I20260928 14:38:09.011652 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 25608
I20260928 14:38:09.011739 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:09.011934 22581 external_mini_cluster.cc:1428] Setting key eceb904f4204a9eac2b70f7c32f1df3a
I20260928 14:38:09.012432 25676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7be41b68634382813168be93ff292c" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46623 } }
I20260928 14:38:09.012705 25676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:09.012766 25676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7be41b68634382813168be93ff292c, State: Initialized, Role: FOLLOWER
I20260928 14:38:09.012910 25676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7be41b68634382813168be93ff292c" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46623 } }
I20260928 14:38:09.012992 25676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:09.013039 25676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:09.013094 25676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:09.014058 25676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7be41b68634382813168be93ff292c" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46623 } }
I20260928 14:38:09.014190 25676 leader_election.cc:302] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e7be41b68634382813168be93ff292c; no voters:
I20260928 14:38:09.014415 25676 leader_election.cc:288] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:09.014453 25681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:09.014571 25681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [term 1 LEADER]: Becoming Leader. State: Replica: 1e7be41b68634382813168be93ff292c, State: Running, Role: LEADER
I20260928 14:38:09.014729 25676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:09.014714 25681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7be41b68634382813168be93ff292c" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46623 } }
I20260928 14:38:09.015188 25682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e7be41b68634382813168be93ff292c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7be41b68634382813168be93ff292c" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46623 } } }
I20260928 14:38:09.015282 25682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:09.015492 25683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e7be41b68634382813168be93ff292c. Latest consensus state: current_term: 1 leader_uuid: "1e7be41b68634382813168be93ff292c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e7be41b68634382813168be93ff292c" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46623 } } }
I20260928 14:38:09.015812 25683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:09.015969 25688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:09.016582 25688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:09.018272 25688 catalog_manager.cc:1347] Generated new cluster ID: f7c82cad77d54f99a71aa619b139dd49
I20260928 14:38:09.018333 25688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:38:09 dist-test-slave-wwwb krb5kdc[25595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:09.023854 25680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:09.012517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35032 (local address 127.22.13.126:46623)
0928 14:38:09.012706 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:09.012717 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:09.012761 (+ 44us) server_negotiation.cc:407] Connection header received
0928 14:38:09.012930 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:09.012944 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:09.013011 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:09.013159 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:09.013821 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:09.015151 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:09.016498 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:09.016990 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:09.017126 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:09.019852 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:09.019857 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:09.019857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:09.019871 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:09.021682 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:09.022177 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:09.022181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:09.022181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:09.022689 (+ 508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:09.022926 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:09.022927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:09.022928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:09.023206 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:09.023399 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:09.023478 (+ 79us) server_negotiation.cc:299] Negotiation successful
0928 14:38:09.023636 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
I20260928 14:38:09.042274 25688 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:09.042870 25688 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:09.045969 25688 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e7be41b68634382813168be93ff292c: Generated new TSK 0
I20260928 14:38:09.046193 25688 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:38:09.081212 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46623
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:09.168874 25702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:09.169086 25702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:09.169111 25702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:09.170419 25702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:09.170466 25702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:09.170526 25702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:38:09.172017 25702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:09.172299 25702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:09.172554 25702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:09.177706 25707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:09.177692 25710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:09.177786 25708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:09.177842 25702 server_base.cc:1034] running on GCE node
I20260928 14:38:09.178246 25702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:09.178508 25702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:09.179644 25702 hybrid_clock.cc:648] HybridClock initialized: now 1790606289179620 us; error 41 us; skew 500 ppm
Sep 28 14:38:09 dist-test-slave-wwwb krb5kdc[25595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606289, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:09.182305 25702 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:38:09.182700 25702 webserver.cc:460] Webserver started at http://127.22.13.65:46395/ using document root <none> and password file <none>
I20260928 14:38:09.182919 25702 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:09.182963 25702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:09.183059 25702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:09.184109 25702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "1aa2a913c4ff4edb86cede43e3f1d4ce"
format_stamp: "Formatted at 2026-09-28 14:38:09 on dist-test-slave-wwwb"
server_key: "b2167a786a4db8f386c8bf98cad8008a"
server_key_iv: "47c9b9f814eeeeb67625147e313221a1"
server_key_version: "encryptionkey@0"
I20260928 14:38:09.184410 25702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "1aa2a913c4ff4edb86cede43e3f1d4ce"
format_stamp: "Formatted at 2026-09-28 14:38:09 on dist-test-slave-wwwb"
server_key: "b2167a786a4db8f386c8bf98cad8008a"
server_key_iv: "47c9b9f814eeeeb67625147e313221a1"
server_key_version: "encryptionkey@0"
I20260928 14:38:09.185762 25702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 14:38:09.186699 25717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:09.186882 25702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:09.186981 25702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "1aa2a913c4ff4edb86cede43e3f1d4ce"
format_stamp: "Formatted at 2026-09-28 14:38:09 on dist-test-slave-wwwb"
server_key: "b2167a786a4db8f386c8bf98cad8008a"
server_key_iv: "47c9b9f814eeeeb67625147e313221a1"
server_key_version: "encryptionkey@0"
I20260928 14:38:09.187048 25702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:09.198819 25702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:09.200836 25702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:09.200965 25702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:09.201186 25702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:09.201455 25702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:09.201489 25702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:09.201516 25702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:09.201531 25702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:09.207940 25702 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:36747
I20260928 14:38:09.207978 25830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:36747 every 8 connection(s)
I20260928 14:38:09.208370 25702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
Sep 28 14:38:09 dist-test-slave-wwwb krb5kdc[25595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606289, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:09.217514 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 25702
I20260928 14:38:09.217633 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:38:09.217845 22581 external_mini_cluster.cc:1428] Setting key 983c5052406792d9ace295b2e0f22aa0
I20260928 14:38:09.218230 25680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:09.209193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:59981 (local address 127.22.13.126:46623)
0928 14:38:09.209272 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:09.209274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:09.209464 (+ 190us) server_negotiation.cc:407] Connection header received
0928 14:38:09.209991 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:09.209992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:09.210009 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:09.210089 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:09.211056 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:09.211601 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:09.212602 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:09.212808 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:09.212870 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:09.215164 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:09.215166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:09.215166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:09.215175 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:09.216674 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:09.217073 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:09.217074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:09.217074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:09.217138 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:09.217388 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:09.217388 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:09.217389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:09.217507 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:09.217564 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:09.217968 (+ 404us) server_negotiation.cc:299] Negotiation successful
0928 14:38:09.218107 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 14:38:09.218393 25833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:09.209109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:46623 (local address 127.22.13.65:59981)
0928 14:38:09.209328 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:09.209372 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:09.209855 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:09.210164 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:09.210167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:09.210746 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:09.210954 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:09.210959 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:09.211691 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:09.211691 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:09.212508 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:09.212510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:09.212887 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:09.212888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:09.212993 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:09.213546 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:09.213555 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:09.215064 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:09.216758 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:09.216766 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:09.216768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:09.217001 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:09.217199 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:09.217200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:09.217200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:09.217323 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:09.217574 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:09.217581 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:09.217777 (+ 196us) client_negotiation.cc:769] Sending connection context
0928 14:38:09.217904 (+ 127us) client_negotiation.cc:240] Negotiation successful
0928 14:38:09.218110 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":99,"threads_started":1}
I20260928 14:38:09.218999 25831 heartbeater.cc:347] Connected to a master server at 127.22.13.126:46623
I20260928 14:38:09.219092 25831 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:09.219278 25831 heartbeater.cc:511] Master 127.22.13.126:46623 requested a full tablet report, sending...
I20260928 14:38:09.219775 25640 ts_manager.cc:194] Registered new tserver with Master: 1aa2a913c4ff4edb86cede43e3f1d4ce (127.22.13.65:36747)
I20260928 14:38:09.220703 25640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:59981
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:38:09.244364 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46623
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:09.327749 25836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:09.327981 25836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:09.328006 25836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:09.329151 25836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:09.329190 25836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:09.329217 25836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:38:09.330703 25836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:09.330910 25836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:09.331099 25836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:09.336361 25844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:09.336377 25842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:09.336441 25841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:09.336635 25836 server_base.cc:1034] running on GCE node
I20260928 14:38:09.336964 25836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:09.337224 25836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:09.338392 25836 hybrid_clock.cc:648] HybridClock initialized: now 1790606289338375 us; error 41 us; skew 500 ppm
Sep 28 14:38:09 dist-test-slave-wwwb krb5kdc[25595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606289, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:09.340982 25836 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:38:09.341445 25836 webserver.cc:460] Webserver started at http://127.22.13.66:43941/ using document root <none> and password file <none>
I20260928 14:38:09.341691 25836 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:09.341764 25836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:09.341924 25836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:09.343152 25836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "425f1cdbec694abfa0e5b634b2db575e"
format_stamp: "Formatted at 2026-09-28 14:38:09 on dist-test-slave-wwwb"
server_key: "fbbce1c2b6b906fafe37622c73b087f8"
server_key_iv: "6084b0d42f17571499cb36fbc9fe1779"
server_key_version: "encryptionkey@0"
I20260928 14:38:09.343549 25836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "425f1cdbec694abfa0e5b634b2db575e"
format_stamp: "Formatted at 2026-09-28 14:38:09 on dist-test-slave-wwwb"
server_key: "fbbce1c2b6b906fafe37622c73b087f8"
server_key_iv: "6084b0d42f17571499cb36fbc9fe1779"
server_key_version: "encryptionkey@0"
I20260928 14:38:09.345027 25836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:38:09.345788 25851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:09.346002 25836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:38:09.346117 25836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "425f1cdbec694abfa0e5b634b2db575e"
format_stamp: "Formatted at 2026-09-28 14:38:09 on dist-test-slave-wwwb"
server_key: "fbbce1c2b6b906fafe37622c73b087f8"
server_key_iv: "6084b0d42f17571499cb36fbc9fe1779"
server_key_version: "encryptionkey@0"
I20260928 14:38:09.346210 25836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:09.357695 25836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:09.359754 25836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:09.359943 25836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:09.360205 25836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:09.360545 25836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:09.360612 25836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:09.360679 25836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:09.360713 25836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:09.367440 25836 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:44449
I20260928 14:38:09.367489 25964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:44449 every 8 connection(s)
I20260928 14:38:09.367862 25836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:38:09.369308 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 25836
I20260928 14:38:09.369441 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:38:09.369665 22581 external_mini_cluster.cc:1428] Setting key d196cbe89c932cd0d41d4806599aadd2
Sep 28 14:38:09 dist-test-slave-wwwb krb5kdc[25595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606289, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
I20260928 14:38:09.379633 25680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:09.368636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:50185 (local address 127.22.13.126:46623)
0928 14:38:09.368712 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:09.368714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:09.368932 (+ 218us) server_negotiation.cc:407] Connection header received
0928 14:38:09.369574 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:09.369576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:09.369590 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:09.369668 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:09.370883 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:09.371661 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:09.373332 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:09.373530 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:09.373586 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:09.376338 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:09.376341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:09.376341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:09.376352 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:09.378133 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:09.378567 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:09.378568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:09.378569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:09.378648 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:09.378919 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:09.378920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:09.378921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:09.379088 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:09.379172 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:09.379444 (+ 272us) server_negotiation.cc:299] Negotiation successful
0928 14:38:09.379536 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:38:09.379855 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:09.368572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:46623 (local address 127.22.13.66:50185)
0928 14:38:09.368793 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:09.368834 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:09.369419 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:09.369756 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:09.369761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:09.370556 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:09.370776 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:09.370781 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:09.371795 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:09.371797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:09.373230 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:09.373234 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:09.373634 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:09.373635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:09.373804 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:09.374508 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:09.374521 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:09.376217 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:09.378232 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:09.378239 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:09.378241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:09.378483 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:09.378730 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:09.378731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:09.378731 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:09.378840 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:09.379164 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:09.379168 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:09.379293 (+ 125us) client_negotiation.cc:769] Sending connection context
0928 14:38:09.379421 (+ 128us) client_negotiation.cc:240] Negotiation successful
0928 14:38:09.379594 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":90,"threads_started":1}
I20260928 14:38:09.380626 25965 heartbeater.cc:347] Connected to a master server at 127.22.13.126:46623
I20260928 14:38:09.380740 25965 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:09.380927 25965 heartbeater.cc:511] Master 127.22.13.126:46623 requested a full tablet report, sending...
I20260928 14:38:09.381383 25640 ts_manager.cc:194] Registered new tserver with Master: 425f1cdbec694abfa0e5b634b2db575e (127.22.13.66:44449)
I20260928 14:38:09.381755 25640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:50185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:38:09.396754 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46623
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:09.480085 25970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:09.480358 25970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:09.480386 25970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:09.481695 25970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:09.481750 25970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:09.481779 25970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:38:09.483417 25970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:09.483701 25970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:09.483951 25970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:09.489038 25976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:09.489078 25975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:09.489243 25970 server_base.cc:1034] running on GCE node
W20260928 14:38:09.489038 25978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:09.489531 25970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:09.489825 25970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:09.490989 25970 hybrid_clock.cc:648] HybridClock initialized: now 1790606289490975 us; error 44 us; skew 500 ppm
Sep 28 14:38:09 dist-test-slave-wwwb krb5kdc[25595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606289, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:09.493476 25970 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:38:09.494040 25970 webserver.cc:460] Webserver started at http://127.22.13.67:39713/ using document root <none> and password file <none>
I20260928 14:38:09.494287 25970 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:09.494357 25970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:09.494518 25970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:09.495615 25970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "9f5fe390b3e644cc87cfc3cfcb184f40"
format_stamp: "Formatted at 2026-09-28 14:38:09 on dist-test-slave-wwwb"
server_key: "831d84a275dcdb579333def841f8f92d"
server_key_iv: "8b02505d424f46472a95ea084c03456b"
server_key_version: "encryptionkey@0"
I20260928 14:38:09.495995 25970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "9f5fe390b3e644cc87cfc3cfcb184f40"
format_stamp: "Formatted at 2026-09-28 14:38:09 on dist-test-slave-wwwb"
server_key: "831d84a275dcdb579333def841f8f92d"
server_key_iv: "8b02505d424f46472a95ea084c03456b"
server_key_version: "encryptionkey@0"
I20260928 14:38:09.497342 25970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:09.498183 25985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:09.498370 25970 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:09.498474 25970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "9f5fe390b3e644cc87cfc3cfcb184f40"
format_stamp: "Formatted at 2026-09-28 14:38:09 on dist-test-slave-wwwb"
server_key: "831d84a275dcdb579333def841f8f92d"
server_key_iv: "8b02505d424f46472a95ea084c03456b"
server_key_version: "encryptionkey@0"
I20260928 14:38:09.498561 25970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:09.530251 25970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:09.532373 25970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:09.532574 25970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:09.532832 25970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:09.533214 25970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:09.533284 25970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:09.533329 25970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:09.533357 25970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:09.539463 25970 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:34055
I20260928 14:38:09.539507 26098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:34055 every 8 connection(s)
I20260928 14:38:09.539872 25970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:38:09.541755 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 25970
I20260928 14:38:09.541863 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:38:09.542090 22581 external_mini_cluster.cc:1428] Setting key a937ae885ff6f17db919f4d26bd2d307
Sep 28 14:38:09 dist-test-slave-wwwb krb5kdc[25595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606289, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:09.549863 25680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:09.540665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:43857 (local address 127.22.13.126:46623)
0928 14:38:09.540715 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:09.540718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:09.540980 (+ 262us) server_negotiation.cc:407] Connection header received
0928 14:38:09.541459 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:09.541460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:09.541470 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:09.541524 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:09.542925 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:09.543516 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:09.544538 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:09.544744 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:09.544810 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:09.547339 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:09.547340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:09.547341 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:09.547349 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:09.548554 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:09.549026 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:09.549027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:09.549028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:09.549093 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:09.549301 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:09.549302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:09.549302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:09.549411 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:09.549463 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:09.549729 (+ 266us) server_negotiation.cc:299] Negotiation successful
0928 14:38:09.549795 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:38:09.550096 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:09.540659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:46623 (local address 127.22.13.67:43857)
0928 14:38:09.540846 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:09.540883 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:09.541374 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:09.541596 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:09.541600 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:09.542469 (+ 869us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:09.542840 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:09.542846 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:09.543600 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:09.543601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:09.544435 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:09.544437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:09.544831 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:09.544832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:09.544935 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:09.545509 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:09.545518 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:09.547245 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:09.548630 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:09.548636 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:09.548638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:09.548909 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:09.549145 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:09.549145 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:09.549146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:09.549249 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:09.549475 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:09.549480 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:09.549614 (+ 134us) client_negotiation.cc:769] Sending connection context
0928 14:38:09.549702 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 14:38:09.549835 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":100,"threads_started":1}
I20260928 14:38:09.550606 26099 heartbeater.cc:347] Connected to a master server at 127.22.13.126:46623
I20260928 14:38:09.550715 26099 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:09.550904 26099 heartbeater.cc:511] Master 127.22.13.126:46623 requested a full tablet report, sending...
I20260928 14:38:09.551282 25640 ts_manager.cc:194] Registered new tserver with Master: 9f5fe390b3e644cc87cfc3cfcb184f40 (127.22.13.67:34055)
I20260928 14:38:09.551672 25640 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:43857
I20260928 14:38:09.554035 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 14:38:09 dist-test-slave-wwwb krb5kdc[25595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606289, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 28 14:38:09 dist-test-slave-wwwb krb5kdc[25595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606289, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:09.577294 25680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:09.569712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35034 (local address 127.22.13.126:46623)
0928 14:38:09.569792 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:09.569794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:09.569900 (+ 106us) server_negotiation.cc:407] Connection header received
0928 14:38:09.570033 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:09.570034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:09.570048 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:09.570125 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:09.570760 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:09.571266 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:09.572052 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:09.572251 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:09.572332 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:09.574718 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:09.574720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:09.574721 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:09.574737 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:09.575981 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:09.576372 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:09.576373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:09.576374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:09.576442 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:09.576654 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:09.576655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:09.576656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:09.576825 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:09.576936 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:09.577080 (+ 144us) server_negotiation.cc:299] Negotiation successful
0928 14:38:09.577174 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260928 14:38:09.577585 25640 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35034
I20260928 14:38:09.578169 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 25702
I20260928 14:38:09.583277 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 25836
I20260928 14:38:09.587771 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 25970
I20260928 14:38:09.592036 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 25608
2026-09-28T14:38:09Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1855 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:38:09 dist-test-slave-wwwb krb5kdc[26111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:38:09 dist-test-slave-wwwb krb5kdc[26111](info): set up 2 sockets
Sep 28 14:38:09 dist-test-slave-wwwb krb5kdc[26111](info): commencing operation
krb5kdc: starting...
W20260928 14:38:10.630153 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:38:10 dist-test-slave-wwwb krb5kdc[26111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:38:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:38:10.748369 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:38731
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:38731
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:10.833114 26124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:10.833412 26124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:10.833457 26124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:10.834776 26124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:10.834874 26124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:10.834903 26124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:10.834928 26124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:10.834960 26124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:10.836720 26124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:38731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:38731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:10.837047 26124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:10.837311 26124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:10.842609 26132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:10.842612 26130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:10.842643 26129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:10.842777 26124 server_base.cc:1034] running on GCE node
I20260928 14:38:10.843116 26124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:10.843477 26124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:10.844658 26124 hybrid_clock.cc:648] HybridClock initialized: now 1790606290844634 us; error 49 us; skew 500 ppm
Sep 28 14:38:10 dist-test-slave-wwwb krb5kdc[26111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606290, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:10.847471 26124 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:38:10.847985 26124 webserver.cc:460] Webserver started at http://127.22.13.126:35769/ using document root <none> and password file <none>
I20260928 14:38:10.848240 26124 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:10.848292 26124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:10.848474 26124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:10.849653 26124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "f967a3d70f784c0ba6a1896d545ccf4b"
format_stamp: "Formatted at 2026-09-28 14:38:10 on dist-test-slave-wwwb"
server_key: "e4d937b030a926a1c3839176eb03ba3b"
server_key_iv: "be4d0e6136b4ed38d29599c30ddbce13"
server_key_version: "encryptionkey@0"
I20260928 14:38:10.850090 26124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "f967a3d70f784c0ba6a1896d545ccf4b"
format_stamp: "Formatted at 2026-09-28 14:38:10 on dist-test-slave-wwwb"
server_key: "e4d937b030a926a1c3839176eb03ba3b"
server_key_iv: "be4d0e6136b4ed38d29599c30ddbce13"
server_key_version: "encryptionkey@0"
I20260928 14:38:10.851545 26124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:10.852322 26139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:10.852499 26124 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:38:10.852627 26124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "f967a3d70f784c0ba6a1896d545ccf4b"
format_stamp: "Formatted at 2026-09-28 14:38:10 on dist-test-slave-wwwb"
server_key: "e4d937b030a926a1c3839176eb03ba3b"
server_key_iv: "be4d0e6136b4ed38d29599c30ddbce13"
server_key_version: "encryptionkey@0"
I20260928 14:38:10.852713 26124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:10.874302 26124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:10.876416 26124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:10.876597 26124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:10.881055 26124 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:38731
I20260928 14:38:10.881088 26191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:38731 every 8 connection(s)
I20260928 14:38:10.881654 26124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:10.882416 26192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:10.884130 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 26124
I20260928 14:38:10.884266 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:10.884464 22581 external_mini_cluster.cc:1428] Setting key cef31d9a1a830c8be9a9bb5cc1299011
I20260928 14:38:10.885157 26192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b: Bootstrap starting.
I20260928 14:38:10.885996 26192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:10.886359 26192 log.cc:824] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:10.887259 26192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b: No bootstrap required, opened a new log
I20260928 14:38:10.889683 26192 raft_consensus.cc:348] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a3d70f784c0ba6a1896d545ccf4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 38731 } }
I20260928 14:38:10.889882 26192 raft_consensus.cc:374] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:10.889928 26192 raft_consensus.cc:729] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f967a3d70f784c0ba6a1896d545ccf4b, State: Initialized, Role: FOLLOWER
I20260928 14:38:10.890131 26192 consensus_queue.cc:260] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a3d70f784c0ba6a1896d545ccf4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 38731 } }
I20260928 14:38:10.890218 26192 raft_consensus.cc:388] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:10.890268 26192 raft_consensus.cc:482] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:10.890328 26192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:10.890980 26192 raft_consensus.cc:504] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a3d70f784c0ba6a1896d545ccf4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 38731 } }
I20260928 14:38:10.891072 26192 leader_election.cc:302] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f967a3d70f784c0ba6a1896d545ccf4b; no voters:
I20260928 14:38:10.891258 26192 leader_election.cc:288] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:10.891304 26197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:10.891445 26197 raft_consensus.cc:686] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [term 1 LEADER]: Becoming Leader. State: Replica: f967a3d70f784c0ba6a1896d545ccf4b, State: Running, Role: LEADER
I20260928 14:38:10.891543 26192 sys_catalog.cc:564] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:10.891593 26197 consensus_queue.cc:237] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a3d70f784c0ba6a1896d545ccf4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 38731 } }
I20260928 14:38:10.891973 26198 sys_catalog.cc:455] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f967a3d70f784c0ba6a1896d545ccf4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a3d70f784c0ba6a1896d545ccf4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 38731 } } }
I20260928 14:38:10.892053 26198 sys_catalog.cc:458] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [sys.catalog]: This master's current role is: LEADER
Sep 28 14:38:10 dist-test-slave-wwwb krb5kdc[26111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:10.892335 26199 sys_catalog.cc:455] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f967a3d70f784c0ba6a1896d545ccf4b. Latest consensus state: current_term: 1 leader_uuid: "f967a3d70f784c0ba6a1896d545ccf4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f967a3d70f784c0ba6a1896d545ccf4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 38731 } } }
I20260928 14:38:10.892660 26199 sys_catalog.cc:458] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:10.892793 26203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:10.893431 26203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:10.895277 26203 catalog_manager.cc:1347] Generated new cluster ID: c72c8a18ee80454cbd813e121a5b9ac3
I20260928 14:38:10.895335 26203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:38:10.896641 26195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:10.885055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33456 (local address 127.22.13.126:38731)
0928 14:38:10.885269 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:10.885280 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:10.885325 (+ 45us) server_negotiation.cc:407] Connection header received
0928 14:38:10.885985 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:10.885996 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:10.886077 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:10.886253 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:10.887039 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:10.888453 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:10.889205 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:10.889864 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:10.890023 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:10.892909 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:10.892913 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:10.892914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:10.892927 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:10.894841 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:10.895307 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:10.895311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:10.895311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:10.895423 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:10.895655 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:10.895656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:10.895656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:10.895967 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:10.896122 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:10.896291 (+ 169us) server_negotiation.cc:299] Negotiation successful
0928 14:38:10.896430 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":97,"threads_started":1}
I20260928 14:38:10.921624 26203 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:10.922493 26203 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:10.932955 26203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f967a3d70f784c0ba6a1896d545ccf4b: Generated new TSK 0
I20260928 14:38:10.933202 26203 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:38:10.960546 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:38731
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:11.043548 26218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:11.043810 26218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:11.043854 26218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:11.045046 26218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:11.045125 26218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:11.045172 26218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:38:11.046852 26218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:38731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:11.047155 26218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:11.047394 26218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:11.052749 26223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:11.052736 26224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:11.052736 26226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:11.053031 26218 server_base.cc:1034] running on GCE node
I20260928 14:38:11.053267 26218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:11.053473 26218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:11.054615 26218 hybrid_clock.cc:648] HybridClock initialized: now 1790606291054604 us; error 28 us; skew 500 ppm
Sep 28 14:38:11 dist-test-slave-wwwb krb5kdc[26111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606291, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:11.057365 26218 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:38:11.057847 26218 webserver.cc:460] Webserver started at http://127.22.13.65:35189/ using document root <none> and password file <none>
I20260928 14:38:11.058125 26218 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:11.058202 26218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:11.058355 26218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:11.059429 26218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "a5160f7292ad40679636181dc067a660"
format_stamp: "Formatted at 2026-09-28 14:38:11 on dist-test-slave-wwwb"
server_key: "bd760119edf93813fc55987fdb0ca46d"
server_key_iv: "09da3620ad6f032d00baa393f0836a50"
server_key_version: "encryptionkey@0"
I20260928 14:38:11.059823 26218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "a5160f7292ad40679636181dc067a660"
format_stamp: "Formatted at 2026-09-28 14:38:11 on dist-test-slave-wwwb"
server_key: "bd760119edf93813fc55987fdb0ca46d"
server_key_iv: "09da3620ad6f032d00baa393f0836a50"
server_key_version: "encryptionkey@0"
I20260928 14:38:11.061213 26218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:11.062022 26233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:11.062204 26218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:11.062310 26218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "a5160f7292ad40679636181dc067a660"
format_stamp: "Formatted at 2026-09-28 14:38:11 on dist-test-slave-wwwb"
server_key: "bd760119edf93813fc55987fdb0ca46d"
server_key_iv: "09da3620ad6f032d00baa393f0836a50"
server_key_version: "encryptionkey@0"
I20260928 14:38:11.062402 26218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:11.083402 26218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:11.085983 26218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:11.086148 26218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:11.086380 26218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:11.086716 26218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:11.086771 26218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:11.086838 26218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:11.086862 26218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:11.092886 26218 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:36507
I20260928 14:38:11.092962 26346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:36507 every 8 connection(s)
I20260928 14:38:11.093327 26218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:38:11.096318 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 26218
I20260928 14:38:11.096446 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:38:11.096642 22581 external_mini_cluster.cc:1428] Setting key 975c2b33c7d31239d67fb255f1268e47
Sep 28 14:38:11 dist-test-slave-wwwb krb5kdc[26111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606291, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:11.104700 26195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:11.094182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:33445 (local address 127.22.13.126:38731)
0928 14:38:11.094259 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:11.094261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:11.094487 (+ 226us) server_negotiation.cc:407] Connection header received
0928 14:38:11.095518 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:11.095519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:11.095534 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:11.095589 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:11.096453 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:11.097142 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:11.098742 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:11.099060 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:11.099145 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:11.101620 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:11.101621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:11.101622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:11.101632 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:11.103152 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:11.103664 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:11.103665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:11.103665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:11.103718 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:11.104030 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:11.104031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:11.104031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:11.104138 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:11.104189 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:11.104548 (+ 359us) server_negotiation.cc:299] Negotiation successful
0928 14:38:11.104628 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 14:38:11.105028 26349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:11.094147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:38731 (local address 127.22.13.65:33445)
0928 14:38:11.094351 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:11.094396 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:11.095378 (+ 982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:11.095669 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:11.095674 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:11.095964 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:11.096300 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:11.096307 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:11.097241 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:11.097242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:11.098638 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:11.098643 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:11.099165 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:11.099166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:11.099298 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:11.099888 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:11.099897 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:11.101514 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:11.103243 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:11.103252 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:11.103255 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:11.103587 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:11.103784 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:11.103785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:11.103785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:11.103963 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:11.104206 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:11.104213 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:11.104400 (+ 187us) client_negotiation.cc:769] Sending connection context
0928 14:38:11.104534 (+ 134us) client_negotiation.cc:240] Negotiation successful
0928 14:38:11.104727 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":105,"threads_started":1}
I20260928 14:38:11.105650 26347 heartbeater.cc:347] Connected to a master server at 127.22.13.126:38731
I20260928 14:38:11.105772 26347 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
I20260928 14:38:11.106021 26347 heartbeater.cc:511] Master 127.22.13.126:38731 requested a full tablet report, sending...
I20260928 14:38:11.106557 26156 ts_manager.cc:194] Registered new tserver with Master: a5160f7292ad40679636181dc067a660 (127.22.13.65:36507)
I20260928 14:38:11.107301 26156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:33445
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:38:11.124356 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:38731
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:11.207176 26352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:11.207409 26352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:11.207435 26352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:11.208609 26352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:11.208650 26352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:11.208678 26352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:38:11.210363 26352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:38731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:11.210574 26352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:11.210770 26352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:11.215941 26360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:11.215941 26358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:11.216087 26352 server_base.cc:1034] running on GCE node
W20260928 14:38:11.215943 26357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:11.216428 26352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:11.216698 26352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:11.217849 26352 hybrid_clock.cc:648] HybridClock initialized: now 1790606291217836 us; error 28 us; skew 500 ppm
Sep 28 14:38:11 dist-test-slave-wwwb krb5kdc[26111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606291, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:11.220582 26352 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:38:11.221104 26352 webserver.cc:460] Webserver started at http://127.22.13.66:46875/ using document root <none> and password file <none>
I20260928 14:38:11.221361 26352 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:11.221432 26352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:11.221585 26352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:11.222749 26352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "9a840308744a46b6991c9b78b3c5b4b2"
format_stamp: "Formatted at 2026-09-28 14:38:11 on dist-test-slave-wwwb"
server_key: "b88f7f8e6c9c83de103e4d62e3101806"
server_key_iv: "9e71b6b4fd7da6b03a36f8b68d016445"
server_key_version: "encryptionkey@0"
I20260928 14:38:11.223130 26352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "9a840308744a46b6991c9b78b3c5b4b2"
format_stamp: "Formatted at 2026-09-28 14:38:11 on dist-test-slave-wwwb"
server_key: "b88f7f8e6c9c83de103e4d62e3101806"
server_key_iv: "9e71b6b4fd7da6b03a36f8b68d016445"
server_key_version: "encryptionkey@0"
I20260928 14:38:11.224488 26352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:38:11.225235 26367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:11.225404 26352 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:11.225513 26352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "9a840308744a46b6991c9b78b3c5b4b2"
format_stamp: "Formatted at 2026-09-28 14:38:11 on dist-test-slave-wwwb"
server_key: "b88f7f8e6c9c83de103e4d62e3101806"
server_key_iv: "9e71b6b4fd7da6b03a36f8b68d016445"
server_key_version: "encryptionkey@0"
I20260928 14:38:11.225600 26352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:11.239379 26352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:11.241747 26352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:11.241916 26352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:11.242228 26352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:11.242549 26352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:11.242611 26352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:11.242654 26352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:11.242684 26352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:11.248651 26352 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:45449
I20260928 14:38:11.248704 26480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:45449 every 8 connection(s)
I20260928 14:38:11.249258 26352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:38:11.249703 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 26352
I20260928 14:38:11.249818 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:38:11.250070 22581 external_mini_cluster.cc:1428] Setting key 92a555a446b6a9f43a146748c93a322c
Sep 28 14:38:11 dist-test-slave-wwwb krb5kdc[26111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606291, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
I20260928 14:38:11.260406 26195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:11.250025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:40523 (local address 127.22.13.126:38731)
0928 14:38:11.250102 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:11.250105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:11.250448 (+ 343us) server_negotiation.cc:407] Connection header received
0928 14:38:11.251747 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:11.251748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:11.251758 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:11.251801 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:11.252703 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:11.253515 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:11.254653 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:11.254975 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:11.255072 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:11.257354 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:11.257356 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:11.257356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:11.257366 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:11.259010 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:11.259414 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:11.259415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:11.259416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:11.259478 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:11.259711 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:11.259711 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:11.259712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:11.259822 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:11.259875 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:11.260265 (+ 390us) server_negotiation.cc:299] Negotiation successful
0928 14:38:11.260338 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 14:38:11.260439 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:11.250031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:38731 (local address 127.22.13.66:40523)
0928 14:38:11.250315 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:11.250363 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:11.251636 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:11.251880 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:11.251885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:11.252295 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:11.252606 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:11.252612 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:11.253611 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:11.253611 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:11.254546 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:11.254549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:11.255120 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:11.255121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:11.255244 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:11.255830 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:11.255838 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:11.257251 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:11.259096 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:11.259102 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:11.259104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:11.259335 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:11.259554 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:11.259554 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:11.259554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:11.259653 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:11.259879 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:11.259883 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:11.260011 (+ 128us) client_negotiation.cc:769] Sending connection context
0928 14:38:11.260121 (+ 110us) client_negotiation.cc:240] Negotiation successful
0928 14:38:11.260266 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":77,"threads_started":1}
I20260928 14:38:11.260986 26481 heartbeater.cc:347] Connected to a master server at 127.22.13.126:38731
I20260928 14:38:11.261096 26481 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:11.261291 26481 heartbeater.cc:511] Master 127.22.13.126:38731 requested a full tablet report, sending...
I20260928 14:38:11.261721 26156 ts_manager.cc:194] Registered new tserver with Master: 9a840308744a46b6991c9b78b3c5b4b2 (127.22.13.66:45449)
I20260928 14:38:11.262195 26156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:40523
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:38:11.278919 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:38731
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:11.360812 26486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:11.361071 26486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:11.361114 26486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:11.362419 26486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:11.362485 26486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:11.362531 26486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:38:11.364132 26486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:38731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:11.364418 26486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:11.364662 26486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:11.369891 26492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:11.369891 26491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:11.369943 26486 server_base.cc:1034] running on GCE node
W20260928 14:38:11.369891 26494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:11.370419 26486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:11.370639 26486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:11.371783 26486 hybrid_clock.cc:648] HybridClock initialized: now 1790606291371770 us; error 30 us; skew 500 ppm
Sep 28 14:38:11 dist-test-slave-wwwb krb5kdc[26111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606291, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:11.374481 26486 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:38:11.374957 26486 webserver.cc:460] Webserver started at http://127.22.13.67:45499/ using document root <none> and password file <none>
I20260928 14:38:11.375147 26486 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:11.375188 26486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:11.375280 26486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:11.376308 26486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "2c5f472f6c834b4ba9ad7b5228a4c925"
format_stamp: "Formatted at 2026-09-28 14:38:11 on dist-test-slave-wwwb"
server_key: "cfc399ef1a5f73c353bedf54918351d4"
server_key_iv: "517746283af6120d06ff28bdad786f79"
server_key_version: "encryptionkey@0"
I20260928 14:38:11.376610 26486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "2c5f472f6c834b4ba9ad7b5228a4c925"
format_stamp: "Formatted at 2026-09-28 14:38:11 on dist-test-slave-wwwb"
server_key: "cfc399ef1a5f73c353bedf54918351d4"
server_key_iv: "517746283af6120d06ff28bdad786f79"
server_key_version: "encryptionkey@0"
I20260928 14:38:11.377919 26486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 14:38:11.378690 26501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:11.378844 26486 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:38:11.378937 26486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "2c5f472f6c834b4ba9ad7b5228a4c925"
format_stamp: "Formatted at 2026-09-28 14:38:11 on dist-test-slave-wwwb"
server_key: "cfc399ef1a5f73c353bedf54918351d4"
server_key_iv: "517746283af6120d06ff28bdad786f79"
server_key_version: "encryptionkey@0"
I20260928 14:38:11.378995 26486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:11.391213 26486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:11.394164 26486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:11.394331 26486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:11.394539 26486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:11.394798 26486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:11.394829 26486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:11.394851 26486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:11.394865 26486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:11.400755 26486 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:44563
I20260928 14:38:11.400828 26614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:44563 every 8 connection(s)
I20260928 14:38:11.401201 26486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:38:11.403857 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 26486
I20260928 14:38:11.403939 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:38:11.404106 22581 external_mini_cluster.cc:1428] Setting key e5e9b3c5307559e97994f57ebba97bfe
Sep 28 14:38:11 dist-test-slave-wwwb krb5kdc[26111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606291, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:11.411191 26195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:11.402011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:39881 (local address 127.22.13.126:38731)
0928 14:38:11.402079 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:11.402082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:11.402345 (+ 263us) server_negotiation.cc:407] Connection header received
0928 14:38:11.403292 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:11.403293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:11.403304 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:11.403359 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:11.404090 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:11.404825 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:11.405902 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:11.406168 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:11.406254 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:11.408557 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:11.408558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:11.408558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:11.408565 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:11.409829 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:11.410313 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:11.410314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:11.410315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:11.410379 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:11.410635 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:11.410636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:11.410636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:11.410750 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:11.410814 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:11.411058 (+ 244us) server_negotiation.cc:299] Negotiation successful
0928 14:38:11.411121 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 14:38:11.411391 26617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:11.402013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:38731 (local address 127.22.13.67:39881)
0928 14:38:11.402216 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:11.402254 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:11.403195 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:11.403421 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:11.403424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:11.403714 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:11.403948 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:11.403954 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:11.404915 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:11.404916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:11.405802 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:11.405804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:11.406264 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:11.406266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:11.406410 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:11.406987 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:11.406996 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:11.408464 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:11.409926 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:11.409932 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:11.409933 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:11.410232 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:11.410449 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:11.410450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:11.410450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:11.410556 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:11.410816 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:11.410820 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:11.410938 (+ 118us) client_negotiation.cc:769] Sending connection context
0928 14:38:11.411020 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 14:38:11.411146 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":103,"threads_started":1}
I20260928 14:38:11.411895 26615 heartbeater.cc:347] Connected to a master server at 127.22.13.126:38731
I20260928 14:38:11.411978 26615 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:11.412184 26615 heartbeater.cc:511] Master 127.22.13.126:38731 requested a full tablet report, sending...
I20260928 14:38:11.412571 26156 ts_manager.cc:194] Registered new tserver with Master: 2c5f472f6c834b4ba9ad7b5228a4c925 (127.22.13.67:44563)
I20260928 14:38:11.412959 26156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:39881
I20260928 14:38:11.415812 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 14:38:11 dist-test-slave-wwwb krb5kdc[26111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 14:38:11 dist-test-slave-wwwb krb5kdc[26111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.65@KRBTEST.COM
I20260928 14:38:11.438823 26622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:11.431115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46486 (local address 127.22.13.65:36507)
0928 14:38:11.431292 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:11.431300 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:11.431317 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:38:11.431344 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:11.431346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:11.431361 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:11.431431 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:11.431996 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:11.432753 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:11.433554 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:11.433915 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:11.434056 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:11.436244 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:11.436255 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:11.436255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:11.436263 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:11.437747 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:11.438147 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:11.438150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:11.438150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:11.438209 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:11.438365 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:11.438365 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:11.438366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:11.438499 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:11.438602 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:11.438673 (+ 71us) server_negotiation.cc:299] Negotiation successful
0928 14:38:11.438739 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
Sep 28 14:38:11 dist-test-slave-wwwb krb5kdc[26111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:11.448225 26195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:11.440349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33470 (local address 127.22.13.126:38731)
0928 14:38:11.440422 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:11.440424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:11.440453 (+ 29us) server_negotiation.cc:407] Connection header received
0928 14:38:11.440537 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:11.440538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:11.440548 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:11.440604 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:11.441089 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:11.441595 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:11.442380 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:11.442588 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:11.442674 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:11.444914 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:11.444916 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:11.444916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:11.444925 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:11.446469 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:11.447037 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:11.447038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:11.447039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:11.447101 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:11.447312 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:11.447313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:11.447313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:11.447506 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:11.447627 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:11.448052 (+ 425us) server_negotiation.cc:299] Negotiation successful
0928 14:38:11.448149 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260928 14:38:11.448707 26156 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33470
I20260928 14:38:11.449083 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 26218
I20260928 14:38:11.455780 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 26352
I20260928 14:38:11.460994 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 26486
I20260928 14:38:11.465310 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 26124
2026-09-28T14:38:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1873 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:38:11 dist-test-slave-wwwb krb5kdc[26627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:38:11 dist-test-slave-wwwb krb5kdc[26627](info): set up 2 sockets
Sep 28 14:38:11 dist-test-slave-wwwb krb5kdc[26627](info): commencing operation
krb5kdc: starting...
W20260928 14:38:12.494235 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:38:12 dist-test-slave-wwwb krb5kdc[26627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:38:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:38:12.612713 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:40393
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:40393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:12.694204 26640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:12.694473 26640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:12.694527 26640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:12.695730 26640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:12.695802 26640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:12.695827 26640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:12.695858 26640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:12.695905 26640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:12.697502 26640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:40393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:40393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:12.697785 26640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:12.698076 26640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:38:12.702935 26640 server_base.cc:1034] running on GCE node
W20260928 14:38:12.702921 26648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:12.702911 26646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:12.702914 26645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:12.703331 26640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:12.703646 26640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:12.704815 26640 hybrid_clock.cc:648] HybridClock initialized: now 1790606292704803 us; error 45 us; skew 500 ppm
Sep 28 14:38:12 dist-test-slave-wwwb krb5kdc[26627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606292, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:12.707420 26640 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:38:12.707827 26640 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:12.707901 26640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:12.708052 26640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:12.709215 26640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "ddf2d2b3f9bf401aab510fcd0a715ec1"
format_stamp: "Formatted at 2026-09-28 14:38:12 on dist-test-slave-wwwb"
server_key: "6ab61deb57cedfea3a75f7d0e401c57b"
server_key_iv: "d637d24776a3a1308046bbde136c23a7"
server_key_version: "encryptionkey@0"
I20260928 14:38:12.709592 26640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "ddf2d2b3f9bf401aab510fcd0a715ec1"
format_stamp: "Formatted at 2026-09-28 14:38:12 on dist-test-slave-wwwb"
server_key: "6ab61deb57cedfea3a75f7d0e401c57b"
server_key_iv: "d637d24776a3a1308046bbde136c23a7"
server_key_version: "encryptionkey@0"
I20260928 14:38:12.711045 26640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:38:12.711802 26654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:12.711994 26640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:12.712100 26640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "ddf2d2b3f9bf401aab510fcd0a715ec1"
format_stamp: "Formatted at 2026-09-28 14:38:12 on dist-test-slave-wwwb"
server_key: "6ab61deb57cedfea3a75f7d0e401c57b"
server_key_iv: "d637d24776a3a1308046bbde136c23a7"
server_key_version: "encryptionkey@0"
I20260928 14:38:12.712175 26640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:12.729712 26640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:12.731787 26640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:12.731976 26640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:12.736189 26640 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:40393
I20260928 14:38:12.736225 26706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:40393 every 8 connection(s)
I20260928 14:38:12.736531 26640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:12.737469 26707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:12.738034 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 26640
I20260928 14:38:12.738161 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:12.738334 22581 external_mini_cluster.cc:1428] Setting key 409c37c17de4f5c0105fddface2bef51
I20260928 14:38:12.740684 26707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1: Bootstrap starting.
I20260928 14:38:12.741443 26707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:12.741739 26707 log.cc:824] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:12.742585 26707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1: No bootstrap required, opened a new log
I20260928 14:38:12.745031 26707 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf2d2b3f9bf401aab510fcd0a715ec1" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40393 } }
I20260928 14:38:12.745227 26707 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:12.745265 26707 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddf2d2b3f9bf401aab510fcd0a715ec1, State: Initialized, Role: FOLLOWER
I20260928 14:38:12.745404 26707 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf2d2b3f9bf401aab510fcd0a715ec1" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40393 } }
I20260928 14:38:12.745483 26707 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:12.745514 26707 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:12.745559 26707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [term 0 FOLLOWER]: Advancing to term 1
Sep 28 14:38:12 dist-test-slave-wwwb krb5kdc[26627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:12.746304 26707 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf2d2b3f9bf401aab510fcd0a715ec1" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40393 } }
I20260928 14:38:12.746425 26707 leader_election.cc:302] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddf2d2b3f9bf401aab510fcd0a715ec1; no voters:
I20260928 14:38:12.746613 26707 leader_election.cc:288] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:12.746673 26712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:12.746883 26712 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [term 1 LEADER]: Becoming Leader. State: Replica: ddf2d2b3f9bf401aab510fcd0a715ec1, State: Running, Role: LEADER
I20260928 14:38:12.746994 26707 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:12.747028 26712 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf2d2b3f9bf401aab510fcd0a715ec1" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40393 } }
I20260928 14:38:12.747462 26714 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddf2d2b3f9bf401aab510fcd0a715ec1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf2d2b3f9bf401aab510fcd0a715ec1" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40393 } } }
I20260928 14:38:12.747601 26714 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:12.748041 26721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:12.748488 26713 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddf2d2b3f9bf401aab510fcd0a715ec1. Latest consensus state: current_term: 1 leader_uuid: "ddf2d2b3f9bf401aab510fcd0a715ec1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddf2d2b3f9bf401aab510fcd0a715ec1" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40393 } } }
I20260928 14:38:12.748631 26721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:12.748648 26713 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1 [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:12.750170 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:12.738921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36022 (local address 127.22.13.126:40393)
0928 14:38:12.739111 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:12.739121 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:12.739159 (+ 38us) server_negotiation.cc:407] Connection header received
0928 14:38:12.739707 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:12.739716 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:12.739778 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:12.739947 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:12.740522 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:12.741923 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:12.742871 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:12.743510 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:12.743627 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:12.746077 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:12.746081 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:12.746081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:12.746095 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:12.748406 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:12.748880 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:12.748884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:12.748884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:12.749005 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:12.749239 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:12.749240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:12.749240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:12.749499 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:12.749680 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:12.749790 (+ 110us) server_negotiation.cc:299] Negotiation successful
0928 14:38:12.749959 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":76,"threads_started":1}
I20260928 14:38:12.750644 26721 catalog_manager.cc:1347] Generated new cluster ID: 11ac770351e94abcbcab0a0a63a402aa
I20260928 14:38:12.750696 26721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:38:12.768608 26721 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:12.769439 26721 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:12.779301 26721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1: Generated new TSK 0
I20260928 14:38:12.779549 26721 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:38:12.808583 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40393
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:12.893445 26733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:12.893688 26733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:12.893729 26733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:12.894989 26733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:12.895066 26733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:12.895115 26733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:38:12.896703 26733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:12.896986 26733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:12.897226 26733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:12.902251 26739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:12.902382 26741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:12.902251 26738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:12.902909 26733 server_base.cc:1034] running on GCE node
I20260928 14:38:12.903106 26733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:12.903323 26733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:12.904467 26733 hybrid_clock.cc:648] HybridClock initialized: now 1790606292904455 us; error 28 us; skew 500 ppm
Sep 28 14:38:12 dist-test-slave-wwwb krb5kdc[26627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606292, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:12.907313 26733 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:38:12.907696 26733 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:12.907745 26733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:12.907922 26733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:12.909021 26733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "89f4e472036745c8bbca4d59062568b8"
format_stamp: "Formatted at 2026-09-28 14:38:12 on dist-test-slave-wwwb"
server_key: "e4e65ca47814f8715baa3c0c2b658f3e"
server_key_iv: "27ce43268cc0562152281b2f05fa108f"
server_key_version: "encryptionkey@0"
I20260928 14:38:12.909397 26733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "89f4e472036745c8bbca4d59062568b8"
format_stamp: "Formatted at 2026-09-28 14:38:12 on dist-test-slave-wwwb"
server_key: "e4e65ca47814f8715baa3c0c2b658f3e"
server_key_iv: "27ce43268cc0562152281b2f05fa108f"
server_key_version: "encryptionkey@0"
I20260928 14:38:12.910827 26733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:38:12.911578 26747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:12.911763 26733 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:12.911857 26733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "89f4e472036745c8bbca4d59062568b8"
format_stamp: "Formatted at 2026-09-28 14:38:12 on dist-test-slave-wwwb"
server_key: "e4e65ca47814f8715baa3c0c2b658f3e"
server_key_iv: "27ce43268cc0562152281b2f05fa108f"
server_key_version: "encryptionkey@0"
I20260928 14:38:12.911934 26733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:12.925602 26733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:12.927711 26733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:12.927868 26733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:12.928077 26733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:12.928398 26733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:12.928431 26733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:12.928453 26733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:12.928498 26733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:12.934473 26733 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:38149
I20260928 14:38:12.934515 26860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:38149 every 8 connection(s)
I20260928 14:38:12.934810 26733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
Sep 28 14:38:12 dist-test-slave-wwwb krb5kdc[26627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606292, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:12.944128 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 26733
I20260928 14:38:12.944265 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:38:12.944357 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:12.935609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:59305 (local address 127.22.13.126:40393)
0928 14:38:12.935673 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:12.935675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:12.935868 (+ 193us) server_negotiation.cc:407] Connection header received
0928 14:38:12.936349 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:12.936350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:12.936361 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:12.936424 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:12.937387 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:12.937894 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:12.938986 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:12.939189 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:12.939243 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:12.941352 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:12.941354 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:12.941354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:12.941362 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:12.942987 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:12.943436 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:12.943437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:12.943437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:12.943491 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:12.943710 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:12.943711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:12.943711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:12.943821 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:12.943872 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:12.944159 (+ 287us) server_negotiation.cc:299] Negotiation successful
0928 14:38:12.944261 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 14:38:12.944501 22581 external_mini_cluster.cc:1428] Setting key cecc768e523ed25b71801626014fa514
I20260928 14:38:12.944605 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:12.935579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:40393 (local address 127.22.13.65:59305)
0928 14:38:12.935731 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:12.935775 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:12.936264 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:12.936485 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:12.936488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:12.936751 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:12.937291 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:12.937297 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:12.938065 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:12.938065 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:12.938905 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:12.938907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:12.939275 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:12.939276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:12.939377 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:12.939847 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:12.939855 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:12.941237 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:12.943105 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:12.943113 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:12.943115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:12.943367 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:12.943549 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:12.943550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:12.943550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:12.943653 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:12.943876 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:12.943880 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:12.944009 (+ 129us) client_negotiation.cc:769] Sending connection context
0928 14:38:12.944128 (+ 119us) client_negotiation.cc:240] Negotiation successful
0928 14:38:12.944310 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":76,"threads_started":1}
I20260928 14:38:12.945317 26861 heartbeater.cc:347] Connected to a master server at 127.22.13.126:40393
I20260928 14:38:12.945410 26861 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:12.945590 26861 heartbeater.cc:511] Master 127.22.13.126:40393 requested a full tablet report, sending...
I20260928 14:38:12.946264 26670 ts_manager.cc:194] Registered new tserver with Master: 89f4e472036745c8bbca4d59062568b8 (127.22.13.65:38149)
I20260928 14:38:12.947148 26670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:59305
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:38:12.969290 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40393
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:13.051468 26866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:13.051738 26866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:13.051783 26866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:13.053007 26866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:13.053087 26866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:13.053136 26866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:38:13.054864 26866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:13.055202 26866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:13.055483 26866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:13.060528 26874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:13.060688 26866 server_base.cc:1034] running on GCE node
W20260928 14:38:13.060571 26872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:13.060554 26871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:13.061031 26866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:13.061288 26866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:13.062430 26866 hybrid_clock.cc:648] HybridClock initialized: now 1790606293062423 us; error 49 us; skew 500 ppm
Sep 28 14:38:13 dist-test-slave-wwwb krb5kdc[26627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606293, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:13.064994 26866 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:38:13.065357 26866 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:13.065428 26866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:13.065572 26866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:13.066691 26866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "1951183f3b4a4e9f8e8708a970013526"
format_stamp: "Formatted at 2026-09-28 14:38:13 on dist-test-slave-wwwb"
server_key: "77e950fe21300c65fa1a1a406298016e"
server_key_iv: "fa9594d0205891c0378f01443da2cac7"
server_key_version: "encryptionkey@0"
I20260928 14:38:13.067070 26866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "1951183f3b4a4e9f8e8708a970013526"
format_stamp: "Formatted at 2026-09-28 14:38:13 on dist-test-slave-wwwb"
server_key: "77e950fe21300c65fa1a1a406298016e"
server_key_iv: "fa9594d0205891c0378f01443da2cac7"
server_key_version: "encryptionkey@0"
I20260928 14:38:13.068357 26866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:38:13.069056 26880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:13.069238 26866 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:38:13.069343 26866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "1951183f3b4a4e9f8e8708a970013526"
format_stamp: "Formatted at 2026-09-28 14:38:13 on dist-test-slave-wwwb"
server_key: "77e950fe21300c65fa1a1a406298016e"
server_key_iv: "fa9594d0205891c0378f01443da2cac7"
server_key_version: "encryptionkey@0"
I20260928 14:38:13.069417 26866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:13.084587 26866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:13.086844 26866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:13.087018 26866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:13.087263 26866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:13.087600 26866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:13.087656 26866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:13.087702 26866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:13.087734 26866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:13.093681 26866 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:42169
I20260928 14:38:13.093715 26993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:42169 every 8 connection(s)
I20260928 14:38:13.094065 26866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:38:13.100072 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 26866
I20260928 14:38:13.100167 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:38:13.100387 22581 external_mini_cluster.cc:1428] Setting key 5dc37ad40b1a264fd030306a48b22b44
Sep 28 14:38:13 dist-test-slave-wwwb krb5kdc[26627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606293, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:13.104368 26996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.094771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:40393 (local address 127.22.13.66:59105)
0928 14:38:13.094946 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:13.094993 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:13.095518 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:13.095782 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:13.095787 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:13.096076 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:13.096596 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:13.096601 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.097344 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.097346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:13.098285 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:13.098288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.098691 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.098692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:13.098824 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:13.099399 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:13.099408 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:13.101018 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:13.102986 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:13.102993 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:13.102995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:13.103244 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:13.103479 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:13.103480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:13.103480 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:13.103578 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:13.103855 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:13.103859 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:13.103980 (+ 121us) client_negotiation.cc:769] Sending connection context
0928 14:38:13.104063 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 14:38:13.104188 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260928 14:38:13.104372 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.094798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:59105 (local address 127.22.13.126:40393)
0928 14:38:13.094848 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:13.094850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:13.095087 (+ 237us) server_negotiation.cc:407] Connection header received
0928 14:38:13.095636 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:13.095637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:13.095651 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:13.095696 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:13.096726 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.097242 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.098380 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.098601 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.098683 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:13.101145 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:13.101147 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:13.101147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:13.101156 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:13.102882 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:13.103330 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:13.103332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:13.103332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:13.103410 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:13.103629 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:13.103630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:13.103630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:13.103789 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:13.103862 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:13.104121 (+ 259us) server_negotiation.cc:299] Negotiation successful
0928 14:38:13.104228 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:38:13.104913 26994 heartbeater.cc:347] Connected to a master server at 127.22.13.126:40393
I20260928 14:38:13.105028 26994 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:13.105180 26994 heartbeater.cc:511] Master 127.22.13.126:40393 requested a full tablet report, sending...
I20260928 14:38:13.105572 26670 ts_manager.cc:194] Registered new tserver with Master: 1951183f3b4a4e9f8e8708a970013526 (127.22.13.66:42169)
I20260928 14:38:13.106027 26670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:59105
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:38:13.125442 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40393
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:13.207085 26999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:13.207311 26999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:13.207340 26999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:13.208496 26999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:13.208534 26999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:13.208562 26999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:38:13.210112 26999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:13.210316 26999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:13.210500 26999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:13.215732 27005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:13.215867 27004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:13.215732 27007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:13.216009 26999 server_base.cc:1034] running on GCE node
I20260928 14:38:13.216248 26999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:13.216496 26999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:13.217643 26999 hybrid_clock.cc:648] HybridClock initialized: now 1790606293217629 us; error 30 us; skew 500 ppm
Sep 28 14:38:13 dist-test-slave-wwwb krb5kdc[26627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606293, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:13.220306 26999 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:38:13.220719 26999 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:13.220799 26999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:13.220954 26999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:13.222149 26999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "bbc9b68fb84d4110a9f66f1471c4dfde"
format_stamp: "Formatted at 2026-09-28 14:38:13 on dist-test-slave-wwwb"
server_key: "07e725253b576fef42dfb980b2dbecb3"
server_key_iv: "fa8b365de3cb888fc0697d86469b7354"
server_key_version: "encryptionkey@0"
I20260928 14:38:13.222543 26999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "bbc9b68fb84d4110a9f66f1471c4dfde"
format_stamp: "Formatted at 2026-09-28 14:38:13 on dist-test-slave-wwwb"
server_key: "07e725253b576fef42dfb980b2dbecb3"
server_key_iv: "fa8b365de3cb888fc0697d86469b7354"
server_key_version: "encryptionkey@0"
I20260928 14:38:13.223898 26999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:38:13.224646 27013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:13.224859 26999 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:13.224961 26999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "bbc9b68fb84d4110a9f66f1471c4dfde"
format_stamp: "Formatted at 2026-09-28 14:38:13 on dist-test-slave-wwwb"
server_key: "07e725253b576fef42dfb980b2dbecb3"
server_key_iv: "fa8b365de3cb888fc0697d86469b7354"
server_key_version: "encryptionkey@0"
I20260928 14:38:13.225050 26999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:13.258173 26999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:13.260116 26999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:13.260286 26999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:13.260556 26999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:13.261013 26999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:13.261067 26999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:13.261104 26999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:13.261173 26999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:13.267206 26999 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:41821
I20260928 14:38:13.267236 27126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:41821 every 8 connection(s)
I20260928 14:38:13.267534 26999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:38:13.270542 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 26999
I20260928 14:38:13.270634 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:38:13.270819 22581 external_mini_cluster.cc:1428] Setting key 2dcd0f0f117d45c568f593aa98f1c699
Sep 28 14:38:13 dist-test-slave-wwwb krb5kdc[26627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606293, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:13.277341 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.268294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:54645 (local address 127.22.13.126:40393)
0928 14:38:13.268363 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:13.268365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:13.268599 (+ 234us) server_negotiation.cc:407] Connection header received
0928 14:38:13.269144 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:13.269146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:13.269159 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:13.269219 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:13.270344 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.271060 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.272204 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.272532 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.272658 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:13.274945 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:13.274947 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:13.274947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:13.274955 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:13.276147 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:13.276556 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:13.276557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:13.276557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:13.276607 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:13.276817 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:13.276818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:13.276818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:13.276921 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:13.276971 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:13.277212 (+ 241us) server_negotiation.cc:299] Negotiation successful
0928 14:38:13.277277 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 14:38:13.277562 27129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.268276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:40393 (local address 127.22.13.67:54645)
0928 14:38:13.268468 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:13.268506 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:13.269028 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:13.269312 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:13.269315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:13.269602 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:13.270194 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:13.270205 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.271155 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.271156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:13.272061 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:13.272064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.272667 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.272668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:13.272841 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:13.273418 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:13.273427 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:13.274847 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:13.276222 (+ 1375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:13.276228 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:13.276229 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:13.276491 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:13.276661 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:13.276662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:13.276662 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:13.276764 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:13.276972 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:13.276976 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:13.277095 (+ 119us) client_negotiation.cc:769] Sending connection context
0928 14:38:13.277184 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 14:38:13.277307 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":97,"threads_started":1}
I20260928 14:38:13.278120 27127 heartbeater.cc:347] Connected to a master server at 127.22.13.126:40393
I20260928 14:38:13.278234 27127 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:13.278399 27127 heartbeater.cc:511] Master 127.22.13.126:40393 requested a full tablet report, sending...
I20260928 14:38:13.278805 26670 ts_manager.cc:194] Registered new tserver with Master: bbc9b68fb84d4110a9f66f1471c4dfde (127.22.13.67:41821)
I20260928 14:38:13.279253 26670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:54645
I20260928 14:38:13.282567 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:38:13.289775 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.283891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36028 (local address 127.22.13.126:40393)
0928 14:38:13.283964 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:13.283966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:13.284029 (+ 63us) server_negotiation.cc:407] Connection header received
0928 14:38:13.284073 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:13.284074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:13.284083 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:13.284140 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:13.284699 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.285199 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.285998 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.286197 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.286272 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:13.287432 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:13.287433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:13.287434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:13.287442 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:13.288561 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:13.288930 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:13.288932 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:13.288932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:13.288996 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:13.289195 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:13.289196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:13.289196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:13.289357 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:13.289467 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:13.289579 (+ 112us) server_negotiation.cc:299] Negotiation successful
0928 14:38:13.289663 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 14:38:13.291311 26670 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:38:13.292909 26670 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:38:13.300139 27141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.295594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48588 (local address 127.22.13.66:42169)
0928 14:38:13.295880 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:13.295886 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:13.295900 (+ 14us) server_negotiation.cc:407] Connection header received
0928 14:38:13.296026 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:13.296029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:13.296058 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:13.296230 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:13.296489 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.297594 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.298875 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.299638 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.299725 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:13.299798 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:13.300006 (+ 208us) server_negotiation.cc:299] Negotiation successful
0928 14:38:13.300050 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":110,"threads_started":1}
I20260928 14:38:13.300148 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.295425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:42169 (local address 127.0.0.1:48588)
0928 14:38:13.295776 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:13.295824 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:13.295899 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:13.296317 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:13.296318 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:13.296320 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:13.296382 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:13.296385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.297708 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.297709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:13.298778 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:13.298780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.299754 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.299756 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:13.299848 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:13.299888 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 14:38:13.299979 (+ 91us) client_negotiation.cc:240] Negotiation successful
0928 14:38:13.300045 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":31,"thread_start_us":83,"threads_started":1}
I20260928 14:38:13.300359 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.295593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58490 (local address 127.22.13.67:41821)
0928 14:38:13.295979 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:13.295982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:13.295993 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:38:13.296029 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:13.296031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:13.296047 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:13.296141 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:13.296386 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.297561 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.299146 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.299834 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.299904 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:13.299939 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:13.300200 (+ 261us) server_negotiation.cc:299] Negotiation successful
0928 14:38:13.300253 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":91,"threads_started":1}
I20260928 14:38:13.300523 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.295430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:41821 (local address 127.0.0.1:58490)
0928 14:38:13.295800 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:13.295833 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:13.295900 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:13.296155 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:13.296158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:13.296160 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:13.296299 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:13.296301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.297691 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.297692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:13.299045 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:13.299048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.299922 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.299923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:13.299988 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:13.299992 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:38:13.300069 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:38:13.300105 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":71,"thread_start_us":79,"threads_started":1}
I20260928 14:38:13.301206 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.295675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38149 (local address 127.0.0.1:49166)
0928 14:38:13.296260 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:13.296271 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:13.296321 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:13.296620 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:13.296620 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:13.296622 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:13.296671 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:13.296672 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.297574 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.297575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:13.299046 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:13.299048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.300987 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.300988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:13.301050 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:13.301055 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:38:13.301104 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 14:38:13.301146 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260928 14:38:13.301460 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.295793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49166 (local address 127.22.13.65:38149)
0928 14:38:13.295987 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:13.295992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:13.296395 (+ 403us) server_negotiation.cc:407] Connection header received
0928 14:38:13.296468 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:13.296472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:13.296492 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:13.296622 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:13.296751 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.297441 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.299998 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.300815 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.300888 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:13.300953 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:13.301334 (+ 381us) server_negotiation.cc:299] Negotiation successful
0928 14:38:13.301384 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":89,"threads_started":1}
I20260928 14:38:13.301388 26928 tablet_service.cc:1467] Processing CreateTablet for tablet ab117ac329cb4c79ab6ba90ec0e5503b (DEFAULT_TABLE table=test-table [id=187b13ec4f954090a2592294d686f916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:13.301385 27061 tablet_service.cc:1467] Processing CreateTablet for tablet ab117ac329cb4c79ab6ba90ec0e5503b (DEFAULT_TABLE table=test-table [id=187b13ec4f954090a2592294d686f916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:13.301784 27061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab117ac329cb4c79ab6ba90ec0e5503b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:13.301784 26928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab117ac329cb4c79ab6ba90ec0e5503b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:13.302106 26795 tablet_service.cc:1467] Processing CreateTablet for tablet ab117ac329cb4c79ab6ba90ec0e5503b (DEFAULT_TABLE table=test-table [id=187b13ec4f954090a2592294d686f916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:13.302448 26795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab117ac329cb4c79ab6ba90ec0e5503b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:13.305078 27144 tablet_bootstrap.cc:492] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8: Bootstrap starting.
I20260928 14:38:13.305446 27145 tablet_bootstrap.cc:492] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde: Bootstrap starting.
I20260928 14:38:13.306088 27144 tablet_bootstrap.cc:654] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:13.306092 27145 tablet_bootstrap.cc:654] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:13.306110 27146 tablet_bootstrap.cc:492] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526: Bootstrap starting.
I20260928 14:38:13.306728 27146 tablet_bootstrap.cc:654] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:13.306802 27145 log.cc:824] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:13.306802 27144 log.cc:824] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:13.307425 27146 log.cc:824] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:13.307693 27144 tablet_bootstrap.cc:492] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8: No bootstrap required, opened a new log
I20260928 14:38:13.307778 27144 ts_tablet_manager.cc:1397] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260928 14:38:13.307968 27145 tablet_bootstrap.cc:492] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde: No bootstrap required, opened a new log
I20260928 14:38:13.308036 27145 ts_tablet_manager.cc:1397] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:38:13.308192 27146 tablet_bootstrap.cc:492] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526: No bootstrap required, opened a new log
I20260928 14:38:13.308283 27146 ts_tablet_manager.cc:1397] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:38:13.309728 27146 raft_consensus.cc:348] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1951183f3b4a4e9f8e8708a970013526" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 42169 } } peers { permanent_uuid: "bbc9b68fb84d4110a9f66f1471c4dfde" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41821 } } peers { permanent_uuid: "89f4e472036745c8bbca4d59062568b8" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38149 } }
I20260928 14:38:13.309904 27146 raft_consensus.cc:374] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:13.309974 27146 raft_consensus.cc:729] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1951183f3b4a4e9f8e8708a970013526, State: Initialized, Role: FOLLOWER
I20260928 14:38:13.309877 27144 raft_consensus.cc:348] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1951183f3b4a4e9f8e8708a970013526" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 42169 } } peers { permanent_uuid: "bbc9b68fb84d4110a9f66f1471c4dfde" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41821 } } peers { permanent_uuid: "89f4e472036745c8bbca4d59062568b8" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38149 } }
I20260928 14:38:13.309990 27145 raft_consensus.cc:348] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1951183f3b4a4e9f8e8708a970013526" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 42169 } } peers { permanent_uuid: "bbc9b68fb84d4110a9f66f1471c4dfde" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41821 } } peers { permanent_uuid: "89f4e472036745c8bbca4d59062568b8" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38149 } }
I20260928 14:38:13.310065 27144 raft_consensus.cc:374] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:13.310124 27145 raft_consensus.cc:374] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:13.310149 27144 raft_consensus.cc:729] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f4e472036745c8bbca4d59062568b8, State: Initialized, Role: FOLLOWER
I20260928 14:38:13.310161 27145 raft_consensus.cc:729] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbc9b68fb84d4110a9f66f1471c4dfde, State: Initialized, Role: FOLLOWER
I20260928 14:38:13.310107 27146 consensus_queue.cc:260] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1951183f3b4a4e9f8e8708a970013526" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 42169 } } peers { permanent_uuid: "bbc9b68fb84d4110a9f66f1471c4dfde" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41821 } } peers { permanent_uuid: "89f4e472036745c8bbca4d59062568b8" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38149 } }
I20260928 14:38:13.310281 27145 consensus_queue.cc:260] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1951183f3b4a4e9f8e8708a970013526" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 42169 } } peers { permanent_uuid: "bbc9b68fb84d4110a9f66f1471c4dfde" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41821 } } peers { permanent_uuid: "89f4e472036745c8bbca4d59062568b8" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38149 } }
I20260928 14:38:13.310302 27144 consensus_queue.cc:260] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1951183f3b4a4e9f8e8708a970013526" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 42169 } } peers { permanent_uuid: "bbc9b68fb84d4110a9f66f1471c4dfde" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41821 } } peers { permanent_uuid: "89f4e472036745c8bbca4d59062568b8" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38149 } }
I20260928 14:38:13.310439 27146 ts_tablet_manager.cc:1428] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 14:38:13.310570 27145 ts_tablet_manager.cc:1428] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:38:13.310599 26994 heartbeater.cc:503] Master 127.22.13.126:40393 was elected leader, sending a full tablet report...
I20260928 14:38:13.310631 26861 heartbeater.cc:503] Master 127.22.13.126:40393 was elected leader, sending a full tablet report...
I20260928 14:38:13.310708 27127 heartbeater.cc:503] Master 127.22.13.126:40393 was elected leader, sending a full tablet report...
I20260928 14:38:13.311569 27144 ts_tablet_manager.cc:1428] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
W20260928 14:38:13.344537 26995 tablet.cc:2369] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:38:13.435580 26862 tablet.cc:2369] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:38:13.466670 27150 raft_consensus.cc:482] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:38:13.466840 27150 raft_consensus.cc:504] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1951183f3b4a4e9f8e8708a970013526" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 42169 } } peers { permanent_uuid: "bbc9b68fb84d4110a9f66f1471c4dfde" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41821 } } peers { permanent_uuid: "89f4e472036745c8bbca4d59062568b8" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38149 } }
I20260928 14:38:13.467253 27150 leader_election.cc:288] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbc9b68fb84d4110a9f66f1471c4dfde (127.22.13.67:41821), 89f4e472036745c8bbca4d59062568b8 (127.22.13.65:38149)
I20260928 14:38:13.470269 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.467542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:35141 (local address 127.22.13.67:41821)
0928 14:38:13.467595 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:13.467597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:13.467608 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:38:13.467642 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:13.467643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:13.467661 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:13.467746 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:13.467933 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.468453 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.469486 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.469878 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.469985 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:13.469994 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:13.470154 (+ 160us) server_negotiation.cc:299] Negotiation successful
0928 14:38:13.470191 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 14:38:13.470273 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.467529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:58051 (local address 127.22.13.65:38149)
0928 14:38:13.467580 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:13.467583 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:13.467748 (+ 165us) server_negotiation.cc:407] Connection header received
0928 14:38:13.467793 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:13.467794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:13.467804 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:13.467856 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:13.468039 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.468507 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.469520 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.469877 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.469984 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:13.469994 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:13.470154 (+ 160us) server_negotiation.cc:299] Negotiation successful
0928 14:38:13.470190 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:38:13.470273 26996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.467452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:41821 (local address 127.22.13.66:35141)
0928 14:38:13.467524 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:13.467538 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:13.467586 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:13.467770 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:13.467771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:13.467773 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:13.467854 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:13.467856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.468535 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.468536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:13.469383 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:13.469386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.469980 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.469981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:13.470072 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:13.470075 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:38:13.470147 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 14:38:13.470190 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260928 14:38:13.470569 27153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.467452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38149 (local address 127.22.13.66:58051)
0928 14:38:13.467667 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:13.467678 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:13.467736 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:13.467877 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:13.467878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:13.467879 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:13.467969 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:13.467971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.468596 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.468597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:13.469406 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:13.469408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.469991 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.469992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:13.470076 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:13.470080 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:38:13.470148 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 14:38:13.470191 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":38,"thread_start_us":98,"threads_started":1}
I20260928 14:38:13.470722 27081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab117ac329cb4c79ab6ba90ec0e5503b" candidate_uuid: "1951183f3b4a4e9f8e8708a970013526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbc9b68fb84d4110a9f66f1471c4dfde" is_pre_election: true
I20260928 14:38:13.470908 27081 raft_consensus.cc:2454] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1951183f3b4a4e9f8e8708a970013526 in term 0.
I20260928 14:38:13.471011 26815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab117ac329cb4c79ab6ba90ec0e5503b" candidate_uuid: "1951183f3b4a4e9f8e8708a970013526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f4e472036745c8bbca4d59062568b8" is_pre_election: true
I20260928 14:38:13.471186 26815 raft_consensus.cc:2454] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1951183f3b4a4e9f8e8708a970013526 in term 0.
I20260928 14:38:13.471174 26881 leader_election.cc:302] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1951183f3b4a4e9f8e8708a970013526, bbc9b68fb84d4110a9f66f1471c4dfde; no voters:
I20260928 14:38:13.471307 27150 raft_consensus.cc:2781] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:38:13.471390 27150 raft_consensus.cc:482] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:38:13.471415 27150 raft_consensus.cc:3037] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:13.472086 27150 raft_consensus.cc:504] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1951183f3b4a4e9f8e8708a970013526" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 42169 } } peers { permanent_uuid: "bbc9b68fb84d4110a9f66f1471c4dfde" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41821 } } peers { permanent_uuid: "89f4e472036745c8bbca4d59062568b8" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38149 } }
I20260928 14:38:13.472230 27150 leader_election.cc:288] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [CANDIDATE]: Term 1 election: Requested vote from peers bbc9b68fb84d4110a9f66f1471c4dfde (127.22.13.67:41821), 89f4e472036745c8bbca4d59062568b8 (127.22.13.65:38149)
I20260928 14:38:13.472388 27081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab117ac329cb4c79ab6ba90ec0e5503b" candidate_uuid: "1951183f3b4a4e9f8e8708a970013526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbc9b68fb84d4110a9f66f1471c4dfde"
I20260928 14:38:13.472401 26815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab117ac329cb4c79ab6ba90ec0e5503b" candidate_uuid: "1951183f3b4a4e9f8e8708a970013526" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f4e472036745c8bbca4d59062568b8"
I20260928 14:38:13.472466 27081 raft_consensus.cc:3037] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:13.472466 26815 raft_consensus.cc:3037] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:13.473340 26815 raft_consensus.cc:2454] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1951183f3b4a4e9f8e8708a970013526 in term 1.
I20260928 14:38:13.473340 27081 raft_consensus.cc:2454] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1951183f3b4a4e9f8e8708a970013526 in term 1.
I20260928 14:38:13.473481 26881 leader_election.cc:302] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1951183f3b4a4e9f8e8708a970013526, bbc9b68fb84d4110a9f66f1471c4dfde; no voters:
I20260928 14:38:13.473603 27150 raft_consensus.cc:2781] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:13.473769 27150 raft_consensus.cc:686] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [term 1 LEADER]: Becoming Leader. State: Replica: 1951183f3b4a4e9f8e8708a970013526, State: Running, Role: LEADER
I20260928 14:38:13.473881 27150 consensus_queue.cc:237] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1951183f3b4a4e9f8e8708a970013526" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 42169 } } peers { permanent_uuid: "bbc9b68fb84d4110a9f66f1471c4dfde" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41821 } } peers { permanent_uuid: "89f4e472036745c8bbca4d59062568b8" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38149 } }
I20260928 14:38:13.474789 26670 catalog_manager.cc:5495] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1951183f3b4a4e9f8e8708a970013526 (127.22.13.66). New cstate: current_term: 1 leader_uuid: "1951183f3b4a4e9f8e8708a970013526" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1951183f3b4a4e9f8e8708a970013526" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 42169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbc9b68fb84d4110a9f66f1471c4dfde" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89f4e472036745c8bbca4d59062568b8" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38149 } health_report { overall_health: UNKNOWN } } }
I20260928 14:38:13.480444 27141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.478295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48590 (local address 127.22.13.66:42169)
0928 14:38:13.478342 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:13.478344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:13.478368 (+ 24us) server_negotiation.cc:407] Connection header received
0928 14:38:13.478430 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:13.478431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:13.478446 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:13.478503 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:38:13.478613 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.479067 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.479804 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:13.479985 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:13.480036 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:13.480169 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:38:13.480270 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:38:13.480326 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:13.480357 (+ 31us) server_negotiation.cc:299] Negotiation successful
0928 14:38:13.480373 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:38:13.482339 27081 raft_consensus.cc:1261] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde [term 1 FOLLOWER]: Refusing update from remote peer 1951183f3b4a4e9f8e8708a970013526: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:13.482398 26815 raft_consensus.cc:1261] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8 [term 1 FOLLOWER]: Refusing update from remote peer 1951183f3b4a4e9f8e8708a970013526: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:13.482745 27150 consensus_queue.cc:1035] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbc9b68fb84d4110a9f66f1471c4dfde" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:13.482884 27154 consensus_queue.cc:1035] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89f4e472036745c8bbca4d59062568b8" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:13.485805 27158 mvcc.cc:205] Tried to move back new op lower bound from 7334323378100535296 to 7334323378069737472. Current Snapshot: MvccSnapshot[applied={T|T < 7334323378100535296 or (T in {7334323378100535296})}]
I20260928 14:38:13.486160 27159 mvcc.cc:205] Tried to move back new op lower bound from 7334323378100535296 to 7334323378069737472. Current Snapshot: MvccSnapshot[applied={T|T < 7334323378100535296}]
I20260928 14:38:13.488116 26670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:13.488229 26670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:13.488983 26670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ddf2d2b3f9bf401aab510fcd0a715ec1: Sending DeleteTablet for 3 replicas of tablet ab117ac329cb4c79ab6ba90ec0e5503b
I20260928 14:38:13.489317 27061 tablet_service.cc:1514] Processing DeleteTablet for tablet ab117ac329cb4c79ab6ba90ec0e5503b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:13 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:58490
I20260928 14:38:13.489362 26795 tablet_service.cc:1514] Processing DeleteTablet for tablet ab117ac329cb4c79ab6ba90ec0e5503b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:13 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:49166
I20260928 14:38:13.489573 26928 tablet_service.cc:1514] Processing DeleteTablet for tablet ab117ac329cb4c79ab6ba90ec0e5503b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:13 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:48588
I20260928 14:38:13.489665 27168 tablet_replica.cc:330] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8: stopping tablet replica
I20260928 14:38:13.489787 27168 raft_consensus.cc:2229] T ab117ac329cb4c79ab6ba90ec0e5503b P 89f4e472036745c8bbca4d59062568b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:13.489811 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 26733
I20260928 14:38:13.490460 27167 tablet_replica.cc:330] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde: stopping tablet replica
I20260928 14:38:13.490562 27167 raft_consensus.cc:2229] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:13.490660 27167 raft_consensus.cc:2258] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:13.492146 27167 ts_tablet_manager.cc:1905] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:13.494064 27169 tablet_replica.cc:330] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526: stopping tablet replica
I20260928 14:38:13.494230 27167 ts_tablet_manager.cc:1918] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:38:13.494313 27167 log.cc:1196] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/ab117ac329cb4c79ab6ba90ec0e5503b
I20260928 14:38:13.494249 27169 raft_consensus.cc:2229] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:38:13.494526 27169 raft_consensus.cc:2258] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:13.494588 27167 ts_tablet_manager.cc:1939] T ab117ac329cb4c79ab6ba90ec0e5503b P bbc9b68fb84d4110a9f66f1471c4dfde: Deleting consensus metadata
I20260928 14:38:13.495195 26655 catalog_manager.cc:4841] TS bbc9b68fb84d4110a9f66f1471c4dfde (127.22.13.67:41821): tablet ab117ac329cb4c79ab6ba90ec0e5503b (table test-table [id=187b13ec4f954090a2592294d686f916]) successfully deleted
I20260928 14:38:13.495359 27169 ts_tablet_manager.cc:1905] T ab117ac329cb4c79ab6ba90ec0e5503b P 1951183f3b4a4e9f8e8708a970013526: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260928 14:38:13.496080 26657 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:38149 (error 108)
I20260928 14:38:13.496353 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 26866
I20260928 14:38:13.496713 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.496594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38149 (local address 127.0.0.1:49180)
0928 14:38:13.496642 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:13.496673 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:38149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260928 14:38:13.496976 26657 catalog_manager.cc:4584] TS 89f4e472036745c8bbca4d59062568b8 (127.22.13.65:38149): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab117ac329cb4c79ab6ba90ec0e5503b: Network error: Client connection negotiation failed: client connection to 127.22.13.65:38149: connect: Connection refused (error 111)
I20260928 14:38:13.501263 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 26999
I20260928 14:38:13.506299 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.501337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:42169 (local address 127.0.0.1:48604)
0928 14:38:13.506192 (+ 4855us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:13.506243 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.66:42169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4818}
I20260928 14:38:13.507452 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 26640
2026-09-28T14:38:13Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2043 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-28T14:38:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:13Z Disabled control of system clock
I20260928 14:38:13.536345 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:43655
--webserver_interface=127.22.13.126
--webserver_port=0
--builtin_ntp_servers=127.22.13.84:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:43655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 14:38:13.617280 27174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:13.617559 27174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:13.617597 27174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:13.618885 27174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:13.618969 27174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:13.618990 27174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:13.619012 27174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:13.619038 27174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:13.620510 27174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:43655
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:43655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:13.620837 27174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:13.621169 27174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:13.626221 27179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:13.626247 27182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:13.626221 27180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:13.626539 27174 server_base.cc:1034] running on GCE node
I20260928 14:38:13.626816 27174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:13.627169 27174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:13.628337 27174 hybrid_clock.cc:648] HybridClock initialized: now 1790606293628322 us; error 48 us; skew 500 ppm
I20260928 14:38:13.629654 27174 webserver.cc:460] Webserver started at http://127.22.13.126:41251/ using document root <none> and password file <none>
I20260928 14:38:13.629984 27174 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:13.630054 27174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:13.630208 27174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:13.631349 27174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "83633607ea0f4b81b8fb3a0413da1cb4"
format_stamp: "Formatted at 2026-09-28 14:38:13 on dist-test-slave-wwwb"
server_key: "a9929783b9ae20a1515b14349df6280a"
server_key_iv: "604e71eba096dadc61cfd2addd87a663"
server_key_version: "encryptionkey@0"
I20260928 14:38:13.631728 27174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "83633607ea0f4b81b8fb3a0413da1cb4"
format_stamp: "Formatted at 2026-09-28 14:38:13 on dist-test-slave-wwwb"
server_key: "a9929783b9ae20a1515b14349df6280a"
server_key_iv: "604e71eba096dadc61cfd2addd87a663"
server_key_version: "encryptionkey@0"
I20260928 14:38:13.633088 27174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:13.633844 27188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:13.634054 27174 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:13.634159 27174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "83633607ea0f4b81b8fb3a0413da1cb4"
format_stamp: "Formatted at 2026-09-28 14:38:13 on dist-test-slave-wwwb"
server_key: "a9929783b9ae20a1515b14349df6280a"
server_key_iv: "604e71eba096dadc61cfd2addd87a663"
server_key_version: "encryptionkey@0"
I20260928 14:38:13.634260 27174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:13.637771 27174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:13.638121 27174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:13.638289 27174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:13.642304 27174 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:43655
I20260928 14:38:13.642339 27240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:43655 every 8 connection(s)
I20260928 14:38:13.642724 27174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:13.643479 27241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:13.646477 27241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4: Bootstrap starting.
I20260928 14:38:13.647038 27241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:13.647287 27241 log.cc:824] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:13.647944 27241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4: No bootstrap required, opened a new log
I20260928 14:38:13.649499 27241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83633607ea0f4b81b8fb3a0413da1cb4" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43655 } }
I20260928 14:38:13.649646 27241 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:13.649669 27241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83633607ea0f4b81b8fb3a0413da1cb4, State: Initialized, Role: FOLLOWER
I20260928 14:38:13.649819 27241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83633607ea0f4b81b8fb3a0413da1cb4" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43655 } }
I20260928 14:38:13.649906 27241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:13.649977 27241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:13.650032 27241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:13.650568 27241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83633607ea0f4b81b8fb3a0413da1cb4" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43655 } }
I20260928 14:38:13.650658 27241 leader_election.cc:302] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83633607ea0f4b81b8fb3a0413da1cb4; no voters:
I20260928 14:38:13.650791 27241 leader_election.cc:288] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:13.650871 27244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:13.651041 27244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [term 1 LEADER]: Becoming Leader. State: Replica: 83633607ea0f4b81b8fb3a0413da1cb4, State: Running, Role: LEADER
I20260928 14:38:13.651075 27241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:13.651137 27244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83633607ea0f4b81b8fb3a0413da1cb4" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43655 } }
I20260928 14:38:13.651203 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 27174
I20260928 14:38:13.651328 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:13.651476 27245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83633607ea0f4b81b8fb3a0413da1cb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83633607ea0f4b81b8fb3a0413da1cb4" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43655 } } }
I20260928 14:38:13.651520 22581 external_mini_cluster.cc:1428] Setting key 83b8bda993840a8b7b713e1eb7dc0220
I20260928 14:38:13.651467 27244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83633607ea0f4b81b8fb3a0413da1cb4. Latest consensus state: current_term: 1 leader_uuid: "83633607ea0f4b81b8fb3a0413da1cb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83633607ea0f4b81b8fb3a0413da1cb4" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 43655 } } }
I20260928 14:38:13.651574 27244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:13.651563 27245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:13.652058 27251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:13.652576 27251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:13.654565 27251 catalog_manager.cc:1347] Generated new cluster ID: 607217b2a05146d99875f5847e5b1d82
I20260928 14:38:13.654633 27251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:38:13.655139 27257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.652192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34200 (local address 127.22.13.126:43655)
0928 14:38:13.652642 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:13.652651 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:13.652686 (+ 35us) server_negotiation.cc:407] Connection header received
0928 14:38:13.652940 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:13.652949 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:13.653234 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:13.653406 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:13.654518 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:13.654522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:13.654522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:13.654527 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:13.654570 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:13.654586 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:13.654647 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:13.654914 (+ 267us) server_negotiation.cc:299] Negotiation successful
0928 14:38:13.654971 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":66,"threads_started":1}
I20260928 14:38:13.669328 27251 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:13.669529 27251 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:13.673923 27251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4: Generated new TSK 0
I20260928 14:38:13.674206 27251 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:38:13.679435 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:43655
--builtin_ntp_servers=127.22.13.84:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 14:38:13.764755 27265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:13.764976 27265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:13.765002 27265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:13.766323 27265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:13.766368 27265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:13.766397 27265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:38:13.767831 27265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:43655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:13.768110 27265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:13.768471 27265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:13.774863 27271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:13.774881 27273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:13.775040 27265 server_base.cc:1034] running on GCE node
W20260928 14:38:13.775185 27270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:13.775470 27265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:13.775745 27265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:13.776903 27265 hybrid_clock.cc:648] HybridClock initialized: now 1790606293776878 us; error 41 us; skew 500 ppm
I20260928 14:38:13.778378 27265 webserver.cc:460] Webserver started at http://127.22.13.65:33605/ using document root <none> and password file <none>
I20260928 14:38:13.778654 27265 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:13.778702 27265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:13.778847 27265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:13.779958 27265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "30b733e5762849f796d3919bcc1d90da"
format_stamp: "Formatted at 2026-09-28 14:38:13 on dist-test-slave-wwwb"
server_key: "3c0a1b304eab46c7b3be2b01c09cc933"
server_key_iv: "4e4f8d87749e7b1d7a5684bbcf6ce345"
server_key_version: "encryptionkey@0"
I20260928 14:38:13.780345 27265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "30b733e5762849f796d3919bcc1d90da"
format_stamp: "Formatted at 2026-09-28 14:38:13 on dist-test-slave-wwwb"
server_key: "3c0a1b304eab46c7b3be2b01c09cc933"
server_key_iv: "4e4f8d87749e7b1d7a5684bbcf6ce345"
server_key_version: "encryptionkey@0"
I20260928 14:38:13.781744 27265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260928 14:38:13.782671 27279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:13.782876 27265 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:13.782974 27265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "30b733e5762849f796d3919bcc1d90da"
format_stamp: "Formatted at 2026-09-28 14:38:13 on dist-test-slave-wwwb"
server_key: "3c0a1b304eab46c7b3be2b01c09cc933"
server_key_iv: "4e4f8d87749e7b1d7a5684bbcf6ce345"
server_key_version: "encryptionkey@0"
I20260928 14:38:13.783035 27265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:13.786475 27265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:13.786739 27265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:13.786865 27265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:13.787035 27265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:13.787312 27265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:13.787345 27265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:13.787369 27265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:13.787380 27265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:13.793185 27265 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:38817
I20260928 14:38:13.793285 27392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:38817 every 8 connection(s)
I20260928 14:38:13.793653 27265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:38:13.794591 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 27265
I20260928 14:38:13.794696 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:38:13.794902 22581 external_mini_cluster.cc:1428] Setting key 1620311a64816ced9994012beab6e319
I20260928 14:38:13.796038 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:43655
--builtin_ntp_servers=127.22.13.84:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260928 14:38:13.797364 27257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.794445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:45405 (local address 127.22.13.126:43655)
0928 14:38:13.794499 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:13.794501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:13.794823 (+ 322us) server_negotiation.cc:407] Connection header received
0928 14:38:13.795475 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:13.795476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:13.795484 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:13.795538 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:13.797101 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:13.797102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:13.797103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:13.797107 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:13.797123 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:13.797128 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:13.797189 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:13.797293 (+ 104us) server_negotiation.cc:299] Negotiation successful
0928 14:38:13.797308 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:38:13.797619 27395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.794443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:43655 (local address 127.22.13.65:45405)
0928 14:38:13.794657 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:13.794710 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:13.795355 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:13.795612 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:13.795616 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:13.795626 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:13.796924 (+ 1298us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:38:13.796924 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:13.796965 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:38:13.797002 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:13.797208 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:13.797213 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:13.797217 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:38:13.797297 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 14:38:13.797367 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
I20260928 14:38:13.798121 27393 heartbeater.cc:347] Connected to a master server at 127.22.13.126:43655
I20260928 14:38:13.798228 27393 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:13.798394 27393 heartbeater.cc:511] Master 127.22.13.126:43655 requested a full tablet report, sending...
I20260928 14:38:13.798899 27205 ts_manager.cc:194] Registered new tserver with Master: 30b733e5762849f796d3919bcc1d90da (127.22.13.65:38817)
W20260928 14:38:13.879392 27396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:13.879695 27396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:13.879732 27396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:13.880960 27396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:13.881036 27396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:13.881079 27396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:38:13.882715 27396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:43655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:13.882984 27396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:13.883332 27396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:13.888583 27404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:13.888566 27402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:13.888595 27396 server_base.cc:1034] running on GCE node
W20260928 14:38:13.888561 27401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:13.889276 27396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:13.889492 27396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:13.890666 27396 hybrid_clock.cc:648] HybridClock initialized: now 1790606293890651 us; error 44 us; skew 500 ppm
I20260928 14:38:13.891955 27396 webserver.cc:460] Webserver started at http://127.22.13.66:33289/ using document root <none> and password file <none>
I20260928 14:38:13.892220 27396 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:13.892293 27396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:13.892453 27396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:13.893558 27396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "74558dfd1c584cf88a37bbb0411fd8f1"
format_stamp: "Formatted at 2026-09-28 14:38:13 on dist-test-slave-wwwb"
server_key: "3fb347f8566e086dd8fc57a44093ec59"
server_key_iv: "5eb2d724d6e30f6d9ccd9e8e5d8bb711"
server_key_version: "encryptionkey@0"
I20260928 14:38:13.893927 27396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "74558dfd1c584cf88a37bbb0411fd8f1"
format_stamp: "Formatted at 2026-09-28 14:38:13 on dist-test-slave-wwwb"
server_key: "3fb347f8566e086dd8fc57a44093ec59"
server_key_iv: "5eb2d724d6e30f6d9ccd9e8e5d8bb711"
server_key_version: "encryptionkey@0"
I20260928 14:38:13.895318 27396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:13.896051 27410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:13.896234 27396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:13.896332 27396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "74558dfd1c584cf88a37bbb0411fd8f1"
format_stamp: "Formatted at 2026-09-28 14:38:13 on dist-test-slave-wwwb"
server_key: "3fb347f8566e086dd8fc57a44093ec59"
server_key_iv: "5eb2d724d6e30f6d9ccd9e8e5d8bb711"
server_key_version: "encryptionkey@0"
I20260928 14:38:13.896411 27396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:13.899928 27396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:13.900213 27396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:13.900355 27396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:13.900565 27396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:13.900890 27396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:13.900945 27396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:13.900986 27396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:13.901013 27396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:13.906852 27396 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:44609
I20260928 14:38:13.906891 27523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:44609 every 8 connection(s)
I20260928 14:38:13.907248 27396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:38:13.910521 27257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.908000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:33037 (local address 127.22.13.126:43655)
0928 14:38:13.908063 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:13.908065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:13.908288 (+ 223us) server_negotiation.cc:407] Connection header received
0928 14:38:13.908821 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:13.908823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:13.908829 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:13.908891 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:13.910224 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:13.910225 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:13.910226 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:13.910230 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:13.910257 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:13.910262 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:13.910336 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:13.910430 (+ 94us) server_negotiation.cc:299] Negotiation successful
0928 14:38:13.910446 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 14:38:13.910756 27526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:13.907993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:43655 (local address 127.22.13.66:33037)
0928 14:38:13.908164 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:13.908198 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:13.908691 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:13.908972 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:13.908976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:13.908984 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:13.910054 (+ 1070us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:38:13.910055 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:13.910086 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:38:13.910115 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:13.910333 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:13.910339 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:13.910343 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:38:13.910441 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 14:38:13.910506 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260928 14:38:13.911255 27524 heartbeater.cc:347] Connected to a master server at 127.22.13.126:43655
I20260928 14:38:13.911330 27524 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:13.911492 27524 heartbeater.cc:511] Master 127.22.13.126:43655 requested a full tablet report, sending...
I20260928 14:38:13.911865 27205 ts_manager.cc:194] Registered new tserver with Master: 74558dfd1c584cf88a37bbb0411fd8f1 (127.22.13.66:44609)
I20260928 14:38:13.912077 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 27396
I20260928 14:38:13.912187 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:38:13.912387 22581 external_mini_cluster.cc:1428] Setting key 15996dd27c442247f2d67d8e6ab9c673
I20260928 14:38:13.913228 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:43655
--builtin_ntp_servers=127.22.13.84:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 14:38:13.995151 27527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:13.995373 27527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:13.995397 27527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:13.996606 27527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:13.996657 27527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:13.996690 27527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:38:13.998196 27527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:43655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:13.998442 27527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:13.998769 27527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:14.003882 27535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:14.003885 27533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:14.003878 27532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:14.004199 27527 server_base.cc:1034] running on GCE node
I20260928 14:38:14.004448 27527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:14.004705 27527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:14.005878 27527 hybrid_clock.cc:648] HybridClock initialized: now 1790606294005857 us; error 34 us; skew 500 ppm
I20260928 14:38:14.007180 27527 webserver.cc:460] Webserver started at http://127.22.13.67:37913/ using document root <none> and password file <none>
I20260928 14:38:14.007442 27527 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:14.007530 27527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:14.007681 27527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:14.008802 27527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "bf3b5995221244f8b4332306463b4067"
format_stamp: "Formatted at 2026-09-28 14:38:14 on dist-test-slave-wwwb"
server_key: "5ba81a949c3a26f052c16413e044f8c9"
server_key_iv: "9ff64020e5d45084668239ba194e260d"
server_key_version: "encryptionkey@0"
I20260928 14:38:14.009177 27527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "bf3b5995221244f8b4332306463b4067"
format_stamp: "Formatted at 2026-09-28 14:38:14 on dist-test-slave-wwwb"
server_key: "5ba81a949c3a26f052c16413e044f8c9"
server_key_iv: "9ff64020e5d45084668239ba194e260d"
server_key_version: "encryptionkey@0"
I20260928 14:38:14.010604 27527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:14.011351 27541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:14.011524 27527 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:14.011631 27527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "bf3b5995221244f8b4332306463b4067"
format_stamp: "Formatted at 2026-09-28 14:38:14 on dist-test-slave-wwwb"
server_key: "5ba81a949c3a26f052c16413e044f8c9"
server_key_iv: "9ff64020e5d45084668239ba194e260d"
server_key_version: "encryptionkey@0"
I20260928 14:38:14.011704 27527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:14.015199 27527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:14.015465 27527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:14.015609 27527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:14.015818 27527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:14.016137 27527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:14.016189 27527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:14.016229 27527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:14.016263 27527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:14.021864 27527 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:36599
I20260928 14:38:14.021904 27654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:36599 every 8 connection(s)
I20260928 14:38:14.022305 27527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:38:14.025228 27257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.023084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:60215 (local address 127.22.13.126:43655)
0928 14:38:14.023148 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:14.023150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:14.023451 (+ 301us) server_negotiation.cc:407] Connection header received
0928 14:38:14.023903 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:14.023904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:14.023910 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:14.023964 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:14.025041 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:14.025042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:14.025042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:14.025046 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:14.025060 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:14.025064 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:14.025112 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:14.025159 (+ 47us) server_negotiation.cc:299] Negotiation successful
0928 14:38:14.025173 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 14:38:14.025467 27657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.023080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:43655 (local address 127.22.13.67:60215)
0928 14:38:14.023322 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:14.023361 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:14.023819 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:14.024015 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:14.024019 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:14.024028 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:14.024892 (+ 864us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:38:14.024892 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:14.024922 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:38:14.024950 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:14.025109 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:14.025112 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:14.025115 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:38:14.025166 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 14:38:14.025215 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":120,"threads_started":1}
I20260928 14:38:14.025900 27655 heartbeater.cc:347] Connected to a master server at 127.22.13.126:43655
I20260928 14:38:14.026079 27655 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:14.026242 27655 heartbeater.cc:511] Master 127.22.13.126:43655 requested a full tablet report, sending...
I20260928 14:38:14.026616 27205 ts_manager.cc:194] Registered new tserver with Master: bf3b5995221244f8b4332306463b4067 (127.22.13.67:36599)
I20260928 14:38:14.027930 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 27527
I20260928 14:38:14.028045 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:38:14.028226 22581 external_mini_cluster.cc:1428] Setting key 718230beb6100cda78eb4e39ca6ed2e3
I20260928 14:38:14.029062 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:38:14.031404 27257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.030403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34202 (local address 127.22.13.126:43655)
0928 14:38:14.030472 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:14.030474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:14.030567 (+ 93us) server_negotiation.cc:407] Connection header received
0928 14:38:14.030605 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:14.030606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:14.030610 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:14.030666 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:14.031156 (+ 490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:14.031157 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:14.031157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:14.031160 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:14.031175 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:14.031179 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:14.031227 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:14.031271 (+ 44us) server_negotiation.cc:299] Negotiation successful
0928 14:38:14.031283 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 14:38:14.033051 27205 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:38:14.034680 27205 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:38:14.038844 27663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.036830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38817 (local address 127.0.0.1:43598)
0928 14:38:14.037114 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:14.037158 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:14.037220 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:14.037760 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:14.037761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:14.037764 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:14.038374 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:38:14.038374 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:14.038408 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:38:14.038441 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:14.038685 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:14.038691 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:14.038694 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:38:14.038756 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 14:38:14.038770 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":58,"threads_started":1}
I20260928 14:38:14.038861 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.036914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43598 (local address 127.22.13.65:38817)
0928 14:38:14.037102 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:14.037108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:14.037288 (+ 180us) server_negotiation.cc:407] Connection header received
0928 14:38:14.037320 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:14.037322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:14.037333 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:14.037403 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:14.038539 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:14.038549 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:14.038549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:14.038554 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:14.038591 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:14.038619 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:14.038689 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:14.038770 (+ 81us) server_negotiation.cc:299] Negotiation successful
0928 14:38:14.038785 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
I20260928 14:38:14.038986 27664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.036857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:36599 (local address 127.0.0.1:42916)
0928 14:38:14.037126 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:14.037165 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:14.037225 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:14.037750 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:14.037756 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:14.037762 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:14.038548 (+ 786us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:38:14.038549 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:14.038557 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:38:14.038569 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:14.038844 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:14.038845 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:14.038846 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 14:38:14.038895 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 14:38:14.038909 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":96,"threads_started":1}
I20260928 14:38:14.039232 27666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.036942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42916 (local address 127.22.13.67:36599)
0928 14:38:14.037178 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:14.037183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:14.037214 (+ 31us) server_negotiation.cc:407] Connection header received
0928 14:38:14.037276 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:14.037279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:14.037288 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:14.037382 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:14.038667 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:14.038677 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:14.038677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:14.038682 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:14.038720 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:14.038745 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:14.038811 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:14.039120 (+ 309us) server_negotiation.cc:299] Negotiation successful
0928 14:38:14.039137 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":90,"threads_started":1}
I20260928 14:38:14.039772 27327 tablet_service.cc:1467] Processing CreateTablet for tablet 1e0923041e7b4af298c68533682c2676 (DEFAULT_TABLE table=test-table [id=27da4e593c174b2da9e8ccd35c4e4e5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:14.039832 27589 tablet_service.cc:1467] Processing CreateTablet for tablet 1e0923041e7b4af298c68533682c2676 (DEFAULT_TABLE table=test-table [id=27da4e593c174b2da9e8ccd35c4e4e5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:14.040056 27327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e0923041e7b4af298c68533682c2676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:14.040081 27589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e0923041e7b4af298c68533682c2676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:14.042208 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.037027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:44609 (local address 127.0.0.1:33370)
0928 14:38:14.037482 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:14.037493 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:14.037533 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:14.037772 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:14.037772 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:14.037774 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:14.039207 (+ 1433us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:38:14.039208 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:14.039214 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:38:14.039227 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:14.041999 (+ 2772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:14.042000 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:14.042065 (+ 65us) client_negotiation.cc:769] Sending connection context
0928 14:38:14.042103 (+ 38us) client_negotiation.cc:240] Negotiation successful
0928 14:38:14.042117 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":70,"threads_started":1}
I20260928 14:38:14.042466 27668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.037239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33370 (local address 127.22.13.66:44609)
0928 14:38:14.037419 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:14.037423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:14.037589 (+ 166us) server_negotiation.cc:407] Connection header received
0928 14:38:14.037617 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:14.037620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:14.037630 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:14.037698 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:14.039315 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:14.039322 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:14.039322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:14.039328 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:14.039389 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:14.039412 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:14.042329 (+ 2917us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:14.042362 (+ 33us) server_negotiation.cc:299] Negotiation successful
0928 14:38:14.042381 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260928 14:38:14.043058 27669 tablet_bootstrap.cc:492] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067: Bootstrap starting.
I20260928 14:38:14.043182 27458 tablet_service.cc:1467] Processing CreateTablet for tablet 1e0923041e7b4af298c68533682c2676 (DEFAULT_TABLE table=test-table [id=27da4e593c174b2da9e8ccd35c4e4e5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:14.043571 27458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e0923041e7b4af298c68533682c2676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:14.043792 27669 tablet_bootstrap.cc:654] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:14.044088 27669 log.cc:824] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:14.044473 27670 tablet_bootstrap.cc:492] T 1e0923041e7b4af298c68533682c2676 P 30b733e5762849f796d3919bcc1d90da: Bootstrap starting.
I20260928 14:38:14.044893 27669 tablet_bootstrap.cc:492] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067: No bootstrap required, opened a new log
I20260928 14:38:14.044961 27669 ts_tablet_manager.cc:1397] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:38:14.045162 27670 tablet_bootstrap.cc:654] T 1e0923041e7b4af298c68533682c2676 P 30b733e5762849f796d3919bcc1d90da: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:14.045490 27670 log.cc:824] T 1e0923041e7b4af298c68533682c2676 P 30b733e5762849f796d3919bcc1d90da: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:14.046347 27670 tablet_bootstrap.cc:492] T 1e0923041e7b4af298c68533682c2676 P 30b733e5762849f796d3919bcc1d90da: No bootstrap required, opened a new log
I20260928 14:38:14.046424 27670 ts_tablet_manager.cc:1397] T 1e0923041e7b4af298c68533682c2676 P 30b733e5762849f796d3919bcc1d90da: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:38:14.046854 27673 tablet_bootstrap.cc:492] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1: Bootstrap starting.
I20260928 14:38:14.047430 27673 tablet_bootstrap.cc:654] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:14.047724 27673 log.cc:824] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:14.047720 27669 raft_consensus.cc:348] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b733e5762849f796d3919bcc1d90da" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38817 } } peers { permanent_uuid: "bf3b5995221244f8b4332306463b4067" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 36599 } } peers { permanent_uuid: "74558dfd1c584cf88a37bbb0411fd8f1" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 44609 } }
I20260928 14:38:14.047900 27669 raft_consensus.cc:374] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:14.047943 27669 raft_consensus.cc:729] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf3b5995221244f8b4332306463b4067, State: Initialized, Role: FOLLOWER
I20260928 14:38:14.048062 27669 consensus_queue.cc:260] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b733e5762849f796d3919bcc1d90da" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38817 } } peers { permanent_uuid: "bf3b5995221244f8b4332306463b4067" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 36599 } } peers { permanent_uuid: "74558dfd1c584cf88a37bbb0411fd8f1" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 44609 } }
I20260928 14:38:14.048354 27669 ts_tablet_manager.cc:1428] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260928 14:38:14.048398 27655 heartbeater.cc:503] Master 127.22.13.126:43655 was elected leader, sending a full tablet report...
I20260928 14:38:14.049175 27670 raft_consensus.cc:348] T 1e0923041e7b4af298c68533682c2676 P 30b733e5762849f796d3919bcc1d90da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b733e5762849f796d3919bcc1d90da" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38817 } } peers { permanent_uuid: "bf3b5995221244f8b4332306463b4067" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 36599 } } peers { permanent_uuid: "74558dfd1c584cf88a37bbb0411fd8f1" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 44609 } }
I20260928 14:38:14.049362 27670 raft_consensus.cc:374] T 1e0923041e7b4af298c68533682c2676 P 30b733e5762849f796d3919bcc1d90da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:14.049427 27670 raft_consensus.cc:729] T 1e0923041e7b4af298c68533682c2676 P 30b733e5762849f796d3919bcc1d90da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30b733e5762849f796d3919bcc1d90da, State: Initialized, Role: FOLLOWER
I20260928 14:38:14.049546 27670 consensus_queue.cc:260] T 1e0923041e7b4af298c68533682c2676 P 30b733e5762849f796d3919bcc1d90da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b733e5762849f796d3919bcc1d90da" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38817 } } peers { permanent_uuid: "bf3b5995221244f8b4332306463b4067" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 36599 } } peers { permanent_uuid: "74558dfd1c584cf88a37bbb0411fd8f1" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 44609 } }
I20260928 14:38:14.049822 27670 ts_tablet_manager.cc:1428] T 1e0923041e7b4af298c68533682c2676 P 30b733e5762849f796d3919bcc1d90da: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260928 14:38:14.049860 27393 heartbeater.cc:503] Master 127.22.13.126:43655 was elected leader, sending a full tablet report...
I20260928 14:38:14.050580 27673 tablet_bootstrap.cc:492] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1: No bootstrap required, opened a new log
I20260928 14:38:14.050658 27673 ts_tablet_manager.cc:1397] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260928 14:38:14.052686 27673 raft_consensus.cc:348] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b733e5762849f796d3919bcc1d90da" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38817 } } peers { permanent_uuid: "bf3b5995221244f8b4332306463b4067" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 36599 } } peers { permanent_uuid: "74558dfd1c584cf88a37bbb0411fd8f1" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 44609 } }
I20260928 14:38:14.052814 27673 raft_consensus.cc:374] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:14.052860 27673 raft_consensus.cc:729] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74558dfd1c584cf88a37bbb0411fd8f1, State: Initialized, Role: FOLLOWER
I20260928 14:38:14.052975 27673 consensus_queue.cc:260] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b733e5762849f796d3919bcc1d90da" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38817 } } peers { permanent_uuid: "bf3b5995221244f8b4332306463b4067" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 36599 } } peers { permanent_uuid: "74558dfd1c584cf88a37bbb0411fd8f1" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 44609 } }
I20260928 14:38:14.053268 27673 ts_tablet_manager.cc:1428] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:38:14.053323 27524 heartbeater.cc:503] Master 127.22.13.126:43655 was elected leader, sending a full tablet report...
I20260928 14:38:14.153501 27677 raft_consensus.cc:482] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:38:14.153683 27677 raft_consensus.cc:504] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b733e5762849f796d3919bcc1d90da" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38817 } } peers { permanent_uuid: "bf3b5995221244f8b4332306463b4067" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 36599 } } peers { permanent_uuid: "74558dfd1c584cf88a37bbb0411fd8f1" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 44609 } }
I20260928 14:38:14.154158 27677 leader_election.cc:288] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30b733e5762849f796d3919bcc1d90da (127.22.13.65:38817), bf3b5995221244f8b4332306463b4067 (127.22.13.67:36599)
I20260928 14:38:14.155504 27526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.154346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38817 (local address 127.22.13.66:45187)
0928 14:38:14.154399 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:14.154411 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:14.154453 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:14.154623 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:14.154624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:14.154627 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:14.155120 (+ 493us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:38:14.155121 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:14.155129 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:38:14.155144 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:14.155333 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:14.155335 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:14.155335 (+ 0us) client_negotiation.cc:769] Sending connection context
0928 14:38:14.155412 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:38:14.155427 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 14:38:14.155543 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.154420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:45187 (local address 127.22.13.65:38817)
0928 14:38:14.154474 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:14.154476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:14.154496 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:38:14.154523 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:14.154524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:14.154529 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:14.154616 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:14.155229 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:14.155230 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:14.155231 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:14.155235 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:14.155255 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:14.155261 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:14.155332 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:14.155416 (+ 84us) server_negotiation.cc:299] Negotiation successful
0928 14:38:14.155430 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 14:38:14.155639 27666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.154441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:39497 (local address 127.22.13.67:36599)
0928 14:38:14.154493 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:14.154495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:14.154712 (+ 217us) server_negotiation.cc:407] Connection header received
0928 14:38:14.154781 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:14.154782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:14.154787 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:14.154861 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:14.155392 (+ 531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:14.155393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:14.155394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:14.155399 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:14.155416 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:14.155421 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:14.155477 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:14.155535 (+ 58us) server_negotiation.cc:299] Negotiation successful
0928 14:38:14.155548 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 14:38:14.155840 27678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.154347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:36599 (local address 127.22.13.66:39497)
0928 14:38:14.154636 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:14.154646 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:14.154722 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:14.154865 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:14.154866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:14.154880 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:14.155289 (+ 409us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:38:14.155290 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:14.155297 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:38:14.155311 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:14.155485 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:14.155486 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:14.155486 (+ 0us) client_negotiation.cc:769] Sending connection context
0928 14:38:14.155535 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 14:38:14.155547 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":114,"thread_start_us":121,"threads_started":1}
I20260928 14:38:14.155869 27347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0923041e7b4af298c68533682c2676" candidate_uuid: "74558dfd1c584cf88a37bbb0411fd8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30b733e5762849f796d3919bcc1d90da" is_pre_election: true
I20260928 14:38:14.156008 27347 raft_consensus.cc:2454] T 1e0923041e7b4af298c68533682c2676 P 30b733e5762849f796d3919bcc1d90da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74558dfd1c584cf88a37bbb0411fd8f1 in term 0.
I20260928 14:38:14.156225 27413 leader_election.cc:302] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30b733e5762849f796d3919bcc1d90da, 74558dfd1c584cf88a37bbb0411fd8f1; no voters:
I20260928 14:38:14.156353 27677 raft_consensus.cc:2781] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:38:14.156422 27677 raft_consensus.cc:482] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:38:14.156402 27609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0923041e7b4af298c68533682c2676" candidate_uuid: "74558dfd1c584cf88a37bbb0411fd8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf3b5995221244f8b4332306463b4067" is_pre_election: true
I20260928 14:38:14.156458 27677 raft_consensus.cc:3037] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:14.156543 27609 raft_consensus.cc:2454] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74558dfd1c584cf88a37bbb0411fd8f1 in term 0.
I20260928 14:38:14.157279 27677 raft_consensus.cc:504] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b733e5762849f796d3919bcc1d90da" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38817 } } peers { permanent_uuid: "bf3b5995221244f8b4332306463b4067" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 36599 } } peers { permanent_uuid: "74558dfd1c584cf88a37bbb0411fd8f1" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 44609 } }
I20260928 14:38:14.157436 27677 leader_election.cc:288] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [CANDIDATE]: Term 1 election: Requested vote from peers 30b733e5762849f796d3919bcc1d90da (127.22.13.65:38817), bf3b5995221244f8b4332306463b4067 (127.22.13.67:36599)
I20260928 14:38:14.157575 27609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0923041e7b4af298c68533682c2676" candidate_uuid: "74558dfd1c584cf88a37bbb0411fd8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf3b5995221244f8b4332306463b4067"
I20260928 14:38:14.157567 27347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e0923041e7b4af298c68533682c2676" candidate_uuid: "74558dfd1c584cf88a37bbb0411fd8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30b733e5762849f796d3919bcc1d90da"
I20260928 14:38:14.157645 27609 raft_consensus.cc:3037] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:14.157646 27347 raft_consensus.cc:3037] T 1e0923041e7b4af298c68533682c2676 P 30b733e5762849f796d3919bcc1d90da [term 0 FOLLOWER]: Advancing to term 1
W20260928 14:38:14.157789 27525 tablet.cc:2369] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:38:14.158569 27347 raft_consensus.cc:2454] T 1e0923041e7b4af298c68533682c2676 P 30b733e5762849f796d3919bcc1d90da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74558dfd1c584cf88a37bbb0411fd8f1 in term 1.
I20260928 14:38:14.158569 27609 raft_consensus.cc:2454] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74558dfd1c584cf88a37bbb0411fd8f1 in term 1.
I20260928 14:38:14.158718 27413 leader_election.cc:302] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30b733e5762849f796d3919bcc1d90da, 74558dfd1c584cf88a37bbb0411fd8f1; no voters:
I20260928 14:38:14.158809 27677 raft_consensus.cc:2781] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:14.158936 27677 raft_consensus.cc:686] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [term 1 LEADER]: Becoming Leader. State: Replica: 74558dfd1c584cf88a37bbb0411fd8f1, State: Running, Role: LEADER
I20260928 14:38:14.159025 27677 consensus_queue.cc:237] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b733e5762849f796d3919bcc1d90da" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38817 } } peers { permanent_uuid: "bf3b5995221244f8b4332306463b4067" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 36599 } } peers { permanent_uuid: "74558dfd1c584cf88a37bbb0411fd8f1" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 44609 } }
I20260928 14:38:14.159839 27204 catalog_manager.cc:5495] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74558dfd1c584cf88a37bbb0411fd8f1 (127.22.13.66). New cstate: current_term: 1 leader_uuid: "74558dfd1c584cf88a37bbb0411fd8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30b733e5762849f796d3919bcc1d90da" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf3b5995221244f8b4332306463b4067" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74558dfd1c584cf88a37bbb0411fd8f1" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 44609 } health_report { overall_health: HEALTHY } } }
I20260928 14:38:14.185668 27668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.184557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33378 (local address 127.22.13.66:44609)
0928 14:38:14.184623 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:14.184625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:14.184633 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:38:14.184698 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:14.184700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:14.184713 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:14.184765 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:14.185390 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:14.185392 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:14.185392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:14.185396 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:14.185411 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:14.185415 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:14.185463 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:14.185533 (+ 70us) server_negotiation.cc:299] Negotiation successful
0928 14:38:14.185567 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:38:14.187623 27609 raft_consensus.cc:1261] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067 [term 1 FOLLOWER]: Refusing update from remote peer 74558dfd1c584cf88a37bbb0411fd8f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:14.187676 27347 raft_consensus.cc:1261] T 1e0923041e7b4af298c68533682c2676 P 30b733e5762849f796d3919bcc1d90da [term 1 FOLLOWER]: Refusing update from remote peer 74558dfd1c584cf88a37bbb0411fd8f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:14.187961 27679 consensus_queue.cc:1035] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf3b5995221244f8b4332306463b4067" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:14.188079 27677 consensus_queue.cc:1035] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30b733e5762849f796d3919bcc1d90da" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:14.191170 27683 mvcc.cc:205] Tried to move back new op lower bound from 7334323380989460480 to 7334323380875960320. Current Snapshot: MvccSnapshot[applied={T|T < 7334323380989460480}]
I20260928 14:38:14.192991 27204 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:14.193099 27204 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:14.193888 27204 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83633607ea0f4b81b8fb3a0413da1cb4: Sending DeleteTablet for 3 replicas of tablet 1e0923041e7b4af298c68533682c2676
I20260928 14:38:14.194320 27589 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e0923041e7b4af298c68533682c2676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:14 UTC) from {username='slave'} at 127.0.0.1:42916
I20260928 14:38:14.194325 27327 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e0923041e7b4af298c68533682c2676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:14 UTC) from {username='slave'} at 127.0.0.1:43598
I20260928 14:38:14.194551 27458 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e0923041e7b4af298c68533682c2676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:14 UTC) from {username='slave'} at 127.0.0.1:33370
I20260928 14:38:14.194787 27694 tablet_replica.cc:330] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1: stopping tablet replica
I20260928 14:38:14.194898 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 27265
I20260928 14:38:14.195549 27693 tablet_replica.cc:330] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067: stopping tablet replica
I20260928 14:38:14.195729 27693 raft_consensus.cc:2229] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:14.195945 27693 raft_consensus.cc:2258] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:14.196520 27693 ts_tablet_manager.cc:1905] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:14.194960 27694 raft_consensus.cc:2229] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:38:14.197367 27694 raft_consensus.cc:2258] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:14.197813 27694 ts_tablet_manager.cc:1905] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:14.198465 27693 ts_tablet_manager.cc:1918] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:38:14.198592 27693 log.cc:1196] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/1e0923041e7b4af298c68533682c2676
I20260928 14:38:14.198971 27693 ts_tablet_manager.cc:1939] T 1e0923041e7b4af298c68533682c2676 P bf3b5995221244f8b4332306463b4067: Deleting consensus metadata
I20260928 14:38:14.199889 27694 ts_tablet_manager.cc:1918] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:38:14.199971 27694 log.cc:1196] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/1e0923041e7b4af298c68533682c2676
I20260928 14:38:14.200245 27694 ts_tablet_manager.cc:1939] T 1e0923041e7b4af298c68533682c2676 P 74558dfd1c584cf88a37bbb0411fd8f1: Deleting consensus metadata
I20260928 14:38:14.200559 27192 catalog_manager.cc:4841] TS bf3b5995221244f8b4332306463b4067 (127.22.13.67:36599): tablet 1e0923041e7b4af298c68533682c2676 (table test-table [id=27da4e593c174b2da9e8ccd35c4e4e5a]) successfully deleted
I20260928 14:38:14.200686 27191 catalog_manager.cc:4841] TS 74558dfd1c584cf88a37bbb0411fd8f1 (127.22.13.66:44609): tablet 1e0923041e7b4af298c68533682c2676 (table test-table [id=27da4e593c174b2da9e8ccd35c4e4e5a]) successfully deleted
W20260928 14:38:14.201371 27191 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:38817 (error 108)
I20260928 14:38:14.201498 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 27396
I20260928 14:38:14.202279 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.202127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38817 (local address 127.0.0.1:43606)
0928 14:38:14.202193 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:14.202244 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:38817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260928 14:38:14.202472 27191 catalog_manager.cc:4584] TS 30b733e5762849f796d3919bcc1d90da (127.22.13.65:38817): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e0923041e7b4af298c68533682c2676: Network error: Client connection negotiation failed: client connection to 127.22.13.65:38817: connect: Connection refused (error 111)
I20260928 14:38:14.207203 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 27527
I20260928 14:38:14.211598 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 27174
2026-09-28T14:38:14Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (702 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260928 14:38:14.231418 22581 mini_oidc.cc:152] Starting JWKS server
I20260928 14:38:14.232498 22581 webserver.cc:460] Webserver started at https://127.0.0.1:37971/ using document root <none> and password file <none>
I20260928 14:38:14.232635 22581 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 14:38:14.232842 22581 webserver.cc:460] Webserver started at http://127.0.0.1:40921/ using document root <none> and password file <none>
2026-09-28T14:38:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:14Z Disabled control of system clock
I20260928 14:38:14.240828 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:36273
--webserver_interface=127.22.13.126
--webserver_port=0
--builtin_ntp_servers=127.22.13.84:37981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:36273
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37971/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/testchainca.cert with env {}
W20260928 14:38:14.323103 27701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:14.323418 27701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:14.323473 27701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:14.325245 27701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:14.325300 27701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:14.325314 27701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:14.325327 27701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:14.325371 27701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 14:38:14.325397 27701 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 14:38:14.325419 27701 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37971/jwks.json
I20260928 14:38:14.327121 27701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:37981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:36273
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:36273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37971/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:14.327412 27701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:14.327653 27701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:14.332494 27709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:14.332502 27707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:14.332496 27706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:14.332693 27701 server_base.cc:1034] running on GCE node
I20260928 14:38:14.332953 27701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:14.333285 27701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:14.334457 27701 hybrid_clock.cc:648] HybridClock initialized: now 1790606294334428 us; error 41 us; skew 500 ppm
I20260928 14:38:14.335713 27701 webserver.cc:460] Webserver started at http://127.22.13.126:40069/ using document root <none> and password file <none>
I20260928 14:38:14.335973 27701 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:14.336043 27701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:14.336195 27701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:14.337339 27701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "49e8b7c68a6646038dc06ae9e05f2064"
format_stamp: "Formatted at 2026-09-28 14:38:14 on dist-test-slave-wwwb"
server_key: "5305d544858be7a0c043fe4cb4f1c1d5"
server_key_iv: "6c3e359bd5d3bc6a6973531d30f33a24"
server_key_version: "encryptionkey@0"
I20260928 14:38:14.337730 27701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "49e8b7c68a6646038dc06ae9e05f2064"
format_stamp: "Formatted at 2026-09-28 14:38:14 on dist-test-slave-wwwb"
server_key: "5305d544858be7a0c043fe4cb4f1c1d5"
server_key_iv: "6c3e359bd5d3bc6a6973531d30f33a24"
server_key_version: "encryptionkey@0"
I20260928 14:38:14.339120 27701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:14.339879 27715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:14.340049 27701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:14.340157 27701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "49e8b7c68a6646038dc06ae9e05f2064"
format_stamp: "Formatted at 2026-09-28 14:38:14 on dist-test-slave-wwwb"
server_key: "5305d544858be7a0c043fe4cb4f1c1d5"
server_key_iv: "6c3e359bd5d3bc6a6973531d30f33a24"
server_key_version: "encryptionkey@0"
I20260928 14:38:14.340235 27701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:14.350308 27701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:14.350649 27701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:14.350805 27701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:14.355010 27701 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:36273
I20260928 14:38:14.355041 27767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:36273 every 8 connection(s)
I20260928 14:38:14.355458 27701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:14.356423 27768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:14.358850 27768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064: Bootstrap starting.
I20260928 14:38:14.359476 27768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:14.359789 27768 log.cc:824] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:14.360466 27768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064: No bootstrap required, opened a new log
I20260928 14:38:14.362423 27768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e8b7c68a6646038dc06ae9e05f2064" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36273 } }
I20260928 14:38:14.362574 27768 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:14.362635 27768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e8b7c68a6646038dc06ae9e05f2064, State: Initialized, Role: FOLLOWER
I20260928 14:38:14.362800 27768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e8b7c68a6646038dc06ae9e05f2064" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36273 } }
I20260928 14:38:14.362882 27768 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:14.362931 27768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:14.362982 27768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:14.363533 27768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e8b7c68a6646038dc06ae9e05f2064" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36273 } }
I20260928 14:38:14.363648 27768 leader_election.cc:302] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49e8b7c68a6646038dc06ae9e05f2064; no voters:
I20260928 14:38:14.363845 27768 leader_election.cc:288] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:14.363895 27771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:14.364074 27771 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [term 1 LEADER]: Becoming Leader. State: Replica: 49e8b7c68a6646038dc06ae9e05f2064, State: Running, Role: LEADER
I20260928 14:38:14.364118 27768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:14.364176 27771 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e8b7c68a6646038dc06ae9e05f2064" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36273 } }
I20260928 14:38:14.364746 27772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49e8b7c68a6646038dc06ae9e05f2064" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e8b7c68a6646038dc06ae9e05f2064" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36273 } } }
I20260928 14:38:14.364941 27773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49e8b7c68a6646038dc06ae9e05f2064. Latest consensus state: current_term: 1 leader_uuid: "49e8b7c68a6646038dc06ae9e05f2064" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e8b7c68a6646038dc06ae9e05f2064" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36273 } } }
I20260928 14:38:14.365036 27773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:14.365010 27772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064 [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:14.365279 27782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:14.365541 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 27701
I20260928 14:38:14.365644 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:14.365815 22581 external_mini_cluster.cc:1428] Setting key 792fff6eafa1cd8aea69d4669edbebff
I20260928 14:38:14.366477 27782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:14.368084 27782 catalog_manager.cc:1347] Generated new cluster ID: 765e9cd6c52f404f80963ed5cd45f3aa
I20260928 14:38:14.368140 27782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:38:14.372079 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.366341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44766 (local address 127.22.13.126:36273)
0928 14:38:14.366653 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:14.366662 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:14.366701 (+ 39us) server_negotiation.cc:407] Connection header received
0928 14:38:14.366811 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:14.366816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:14.366879 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:14.367020 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:14.367235 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.368586 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.369693 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.370398 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.370532 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:14.371277 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:14.371281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:14.371281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:14.371533 (+ 252us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:14.371581 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:14.371596 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:14.371671 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:14.371753 (+ 82us) server_negotiation.cc:299] Negotiation successful
0928 14:38:14.371841 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":69,"threads_started":1}
I20260928 14:38:14.373657 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:38:14.373832 27782 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:14.374354 27782 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:14.385682 27782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49e8b7c68a6646038dc06ae9e05f2064: Generated new TSK 0
I20260928 14:38:14.385918 27782 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260928 14:38:14.412559 27798 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260928 14:38:14.412695 27798 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 14:38:14.412875 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.397056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44774 (local address 127.22.13.126:36273)
0928 14:38:14.397105 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:14.397108 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:14.397243 (+ 135us) server_negotiation.cc:407] Connection header received
0928 14:38:14.397313 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:14.397315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:14.397328 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:14.397414 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0928 14:38:14.397577 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.398389 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.399136 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.399445 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.399519 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:14.399717 (+ 198us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 14:38:14.412634 (+ 12917us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37971/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0928 14:38:14.412802 (+ 168us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44774: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37971/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":28}
W20260928 14:38:14.412940 27790 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44774: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37971/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260928 14:38:14.413390 27797 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:14.396960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36273 (local address 127.0.0.1:44774)
0928 14:38:14.397149 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:14.397171 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:14.397232 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:14.397427 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:14.397428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:14.397429 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0928 14:38:14.397496 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:14.397498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.398470 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.398471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:14.399046 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:14.399048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.399529 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.399529 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:14.399578 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:14.399601 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 14:38:14.412919 (+ 13318us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37971/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0928 14:38:14.413087 (+ 168us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.13.126:36273: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37971/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":100,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.22.13.126:36273: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37971/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260928 14:38:14.413883 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 27701
2026-09-28T14:38:14Z chronyd exiting
I20260928 14:38:14.449868 22581 test_util.cc:175] -----------------------------------------------
I20260928 14:38:14.450004 22581 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606270465379-22581-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (219 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260928 14:38:14.450822 22581 mini_oidc.cc:152] Starting JWKS server
I20260928 14:38:14.451509 22581 webserver.cc:460] Webserver started at https://127.0.0.1:40633/ using document root <none> and password file <none>
I20260928 14:38:14.451550 22581 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 14:38:14.451746 22581 webserver.cc:460] Webserver started at http://127.0.0.1:39963/ using document root <none> and password file <none>
2026-09-28T14:38:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:14Z Disabled control of system clock
I20260928 14:38:14.459671 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:39207
--webserver_interface=127.22.13.126
--webserver_port=0
--builtin_ntp_servers=127.22.13.84:41141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:39207
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40633/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/testchainca.cert with env {}
W20260928 14:38:14.541049 27804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:14.541291 27804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:14.541329 27804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:14.542550 27804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:14.542593 27804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:14.542605 27804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:14.542618 27804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:14.542629 27804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 14:38:14.542641 27804 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 14:38:14.542654 27804 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40633/jwks.json
I20260928 14:38:14.544188 27804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:41141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:39207
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:39207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40633/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:14.544538 27804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:14.544806 27804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:38:14.550165 27804 server_base.cc:1034] running on GCE node
W20260928 14:38:14.550132 27810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:14.550109 27809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:14.550130 27812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:14.550627 27804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:14.550966 27804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:14.552134 27804 hybrid_clock.cc:648] HybridClock initialized: now 1790606294552122 us; error 44 us; skew 500 ppm
I20260928 14:38:14.553500 27804 webserver.cc:460] Webserver started at http://127.22.13.126:46553/ using document root <none> and password file <none>
I20260928 14:38:14.553813 27804 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:14.553885 27804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:14.554047 27804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:14.555120 27804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "f5bf83f06a21424bbe600b286b69670d"
format_stamp: "Formatted at 2026-09-28 14:38:14 on dist-test-slave-wwwb"
server_key: "3767ecff99bfe1b79d18a62c11d2d21e"
server_key_iv: "86b270d97ac7b0b337ac22954e5e6893"
server_key_version: "encryptionkey@0"
I20260928 14:38:14.555500 27804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "f5bf83f06a21424bbe600b286b69670d"
format_stamp: "Formatted at 2026-09-28 14:38:14 on dist-test-slave-wwwb"
server_key: "3767ecff99bfe1b79d18a62c11d2d21e"
server_key_iv: "86b270d97ac7b0b337ac22954e5e6893"
server_key_version: "encryptionkey@0"
I20260928 14:38:14.556804 27804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:14.557467 27818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:14.557636 27804 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:14.557734 27804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "f5bf83f06a21424bbe600b286b69670d"
format_stamp: "Formatted at 2026-09-28 14:38:14 on dist-test-slave-wwwb"
server_key: "3767ecff99bfe1b79d18a62c11d2d21e"
server_key_iv: "86b270d97ac7b0b337ac22954e5e6893"
server_key_version: "encryptionkey@0"
I20260928 14:38:14.557821 27804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:14.568192 27804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:14.568543 27804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:14.568706 27804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:14.572800 27804 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:39207
I20260928 14:38:14.572834 27870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:39207 every 8 connection(s)
I20260928 14:38:14.573199 27804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:14.573884 27871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:14.574760 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 27804
I20260928 14:38:14.574880 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:14.575070 22581 external_mini_cluster.cc:1428] Setting key 1d4dc6d5b395cb9db7328c063bf8f834
I20260928 14:38:14.577270 27871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d: Bootstrap starting.
I20260928 14:38:14.578008 27871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:14.578325 27871 log.cc:824] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:14.579079 27871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d: No bootstrap required, opened a new log
I20260928 14:38:14.580760 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.575655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57870 (local address 127.22.13.126:39207)
0928 14:38:14.575852 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:14.575860 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:14.575886 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:38:14.576313 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:14.576321 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:14.576381 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:14.576501 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:14.576756 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.577705 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.578815 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.579298 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.579401 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:14.580070 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:14.580074 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:14.580074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:14.580235 (+ 161us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:14.580278 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:14.580289 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:14.580340 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:14.580443 (+ 103us) server_negotiation.cc:299] Negotiation successful
0928 14:38:14.580533 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":94,"threads_started":1}
I20260928 14:38:14.581390 27871 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5bf83f06a21424bbe600b286b69670d" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39207 } }
I20260928 14:38:14.581563 27871 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:14.581629 27871 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5bf83f06a21424bbe600b286b69670d, State: Initialized, Role: FOLLOWER
I20260928 14:38:14.581773 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:38:14.581766 27871 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5bf83f06a21424bbe600b286b69670d" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39207 } }
I20260928 14:38:14.581851 27871 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:14.581890 27871 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:14.581964 27871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:14.582521 27871 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5bf83f06a21424bbe600b286b69670d" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39207 } }
I20260928 14:38:14.582638 27871 leader_election.cc:302] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5bf83f06a21424bbe600b286b69670d; no voters:
I20260928 14:38:14.582842 27871 leader_election.cc:288] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:14.582870 27876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:14.582980 27876 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [term 1 LEADER]: Becoming Leader. State: Replica: f5bf83f06a21424bbe600b286b69670d, State: Running, Role: LEADER
I20260928 14:38:14.583070 27876 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5bf83f06a21424bbe600b286b69670d" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39207 } }
I20260928 14:38:14.583103 27871 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:14.583496 27877 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5bf83f06a21424bbe600b286b69670d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5bf83f06a21424bbe600b286b69670d" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39207 } } }
I20260928 14:38:14.583511 27878 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5bf83f06a21424bbe600b286b69670d. Latest consensus state: current_term: 1 leader_uuid: "f5bf83f06a21424bbe600b286b69670d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5bf83f06a21424bbe600b286b69670d" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39207 } } }
I20260928 14:38:14.583599 27877 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:14.583619 27878 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d [sys.catalog]: This master's current role is: LEADER
W20260928 14:38:14.584811 27892 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:38:14.584874 27892 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:38:14.584975 27883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:14.585163 27883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:14.586669 27883 catalog_manager.cc:1347] Generated new cluster ID: 265f20840b3a4c8f8cc0be9fc188596a
I20260928 14:38:14.586745 27883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:38:14.600085 27883 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:14.600835 27883 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:14.610870 27883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5bf83f06a21424bbe600b286b69670d: Generated new TSK 0
I20260928 14:38:14.611109 27883 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260928 14:38:14.646220 27901 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260928 14:38:14.646315 27901 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 14:38:14.646616 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.624358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57884 (local address 127.22.13.126:39207)
0928 14:38:14.624434 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:14.624436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:14.624549 (+ 113us) server_negotiation.cc:407] Connection header received
0928 14:38:14.624647 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:14.624648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:14.624660 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:14.624722 (+ 62us) server_negotiation.cc:226] Negotiated authn=JWT
0928 14:38:14.624907 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.625421 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.626597 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.626809 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.626864 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:14.627085 (+ 221us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 14:38:14.646393 (+ 19308us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40633/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0928 14:38:14.646559 (+ 166us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57884: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40633/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":46}
W20260928 14:38:14.646694 27874 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57884: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40633/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260928 14:38:14.646747 27900 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:14.624299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39207 (local address 127.0.0.1:57884)
0928 14:38:14.624469 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:14.624483 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:14.624555 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:14.624750 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:14.624751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:14.624753 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0928 14:38:14.624836 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:14.624839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.625515 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.625516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:14.626498 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:14.626502 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.626899 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.626900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:14.626968 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:14.626975 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 14:38:14.646518 (+ 19543us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40633/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0928 14:38:14.646644 (+ 126us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.13.126:39207: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40633/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":95,"threads_started":1}
I20260928 14:38:14.647151 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 27804
2026-09-28T14:38:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (234 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260928 14:38:14.685060 22581 mini_oidc.cc:152] Starting JWKS server
I20260928 14:38:14.685705 22581 webserver.cc:460] Webserver started at https://127.0.0.1:37539/ using document root <none> and password file <none>
I20260928 14:38:14.685753 22581 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 14:38:14.685901 22581 webserver.cc:460] Webserver started at http://127.0.0.1:43579/ using document root <none> and password file <none>
2026-09-28T14:38:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:14Z Disabled control of system clock
I20260928 14:38:14.693657 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:37851
--webserver_interface=127.22.13.126
--webserver_port=0
--builtin_ntp_servers=127.22.13.84:39693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:37851
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37539/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260928 14:38:14.775768 27907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:14.775997 27907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:14.776021 27907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:14.777155 27907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:14.777194 27907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:14.777205 27907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:14.777218 27907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:14.777231 27907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 14:38:14.777243 27907 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 14:38:14.777254 27907 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37539/jwks.json
I20260928 14:38:14.778604 27907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:39693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:37851
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:37851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37539/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:14.778831 27907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:14.779083 27907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:14.784052 27915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:14.784054 27913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:14.784077 27912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:14.784370 27907 server_base.cc:1034] running on GCE node
I20260928 14:38:14.784600 27907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:14.784897 27907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:14.786026 27907 hybrid_clock.cc:648] HybridClock initialized: now 1790606294786012 us; error 29 us; skew 500 ppm
I20260928 14:38:14.787288 27907 webserver.cc:460] Webserver started at http://127.22.13.126:44541/ using document root <none> and password file <none>
I20260928 14:38:14.787567 27907 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:14.787672 27907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:14.787827 27907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:14.788976 27907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "3b8f4b5a048f466f8af88896595e5ebe"
format_stamp: "Formatted at 2026-09-28 14:38:14 on dist-test-slave-wwwb"
server_key: "0e3a9175cf081fcaf116c8481d5f8331"
server_key_iv: "d0ed408fc78b83ef80d0eb04e49a5263"
server_key_version: "encryptionkey@0"
I20260928 14:38:14.789369 27907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "3b8f4b5a048f466f8af88896595e5ebe"
format_stamp: "Formatted at 2026-09-28 14:38:14 on dist-test-slave-wwwb"
server_key: "0e3a9175cf081fcaf116c8481d5f8331"
server_key_iv: "d0ed408fc78b83ef80d0eb04e49a5263"
server_key_version: "encryptionkey@0"
I20260928 14:38:14.790797 27907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:14.791565 27921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:14.791746 27907 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:14.791852 27907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "3b8f4b5a048f466f8af88896595e5ebe"
format_stamp: "Formatted at 2026-09-28 14:38:14 on dist-test-slave-wwwb"
server_key: "0e3a9175cf081fcaf116c8481d5f8331"
server_key_iv: "d0ed408fc78b83ef80d0eb04e49a5263"
server_key_version: "encryptionkey@0"
I20260928 14:38:14.791952 27907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:14.803951 27907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:14.804293 27907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:14.804454 27907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:14.808493 27907 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:37851
I20260928 14:38:14.808550 27973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:37851 every 8 connection(s)
I20260928 14:38:14.808928 27907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:14.809638 27974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:14.812515 27974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe: Bootstrap starting.
I20260928 14:38:14.813138 27974 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:14.813458 27974 log.cc:824] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:14.814224 27974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe: No bootstrap required, opened a new log
I20260928 14:38:14.815815 27974 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8f4b5a048f466f8af88896595e5ebe" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37851 } }
I20260928 14:38:14.815961 27974 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:14.816010 27974 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b8f4b5a048f466f8af88896595e5ebe, State: Initialized, Role: FOLLOWER
I20260928 14:38:14.816159 27974 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8f4b5a048f466f8af88896595e5ebe" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37851 } }
I20260928 14:38:14.816219 27974 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:14.816244 27974 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:14.816313 27974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:14.816910 27974 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8f4b5a048f466f8af88896595e5ebe" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37851 } }
I20260928 14:38:14.817024 27974 leader_election.cc:302] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b8f4b5a048f466f8af88896595e5ebe; no voters:
I20260928 14:38:14.817224 27974 leader_election.cc:288] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:14.817288 27977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:14.817451 27977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [term 1 LEADER]: Becoming Leader. State: Replica: 3b8f4b5a048f466f8af88896595e5ebe, State: Running, Role: LEADER
I20260928 14:38:14.817485 27974 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:14.817551 27977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8f4b5a048f466f8af88896595e5ebe" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37851 } }
I20260928 14:38:14.817953 27978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b8f4b5a048f466f8af88896595e5ebe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8f4b5a048f466f8af88896595e5ebe" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37851 } } }
I20260928 14:38:14.818043 27978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:14.818212 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 27907
I20260928 14:38:14.818337 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:14.818516 22581 external_mini_cluster.cc:1428] Setting key 2410bb5fe52235e0db3ce2623775a91b
I20260928 14:38:14.819061 27979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b8f4b5a048f466f8af88896595e5ebe. Latest consensus state: current_term: 1 leader_uuid: "3b8f4b5a048f466f8af88896595e5ebe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8f4b5a048f466f8af88896595e5ebe" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37851 } } }
I20260928 14:38:14.819134 27979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:14.819558 27988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:14.820078 27988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:14.821753 27988 catalog_manager.cc:1347] Generated new cluster ID: 39de4da041054467a523d9bdefaacd7b
I20260928 14:38:14.821811 27988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:38:14.824913 27985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.819173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42116 (local address 127.22.13.126:37851)
0928 14:38:14.819372 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:14.819381 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:14.819414 (+ 33us) server_negotiation.cc:407] Connection header received
0928 14:38:14.819554 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:14.819562 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:14.819635 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:14.819780 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:14.820069 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.821446 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.822590 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.823215 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.823656 (+ 441us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:14.824093 (+ 437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:14.824097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:14.824097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:14.824310 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:14.824356 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:14.824370 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:14.824588 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:14.824618 (+ 30us) server_negotiation.cc:299] Negotiation successful
0928 14:38:14.824697 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":95,"threads_started":1}
I20260928 14:38:14.826056 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:38:14.833989 27988 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:14.834517 27988 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:14.840082 27988 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b8f4b5a048f466f8af88896595e5ebe: Generated new TSK 0
I20260928 14:38:14.840291 27988 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:38:14.850833 27985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:14.847085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42130 (local address 127.22.13.126:37851)
0928 14:38:14.847148 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:14.847151 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:14.847251 (+ 100us) server_negotiation.cc:407] Connection header received
0928 14:38:14.847334 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:14.847335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:14.847348 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:14.847414 (+ 66us) server_negotiation.cc:226] Negotiated authn=JWT
0928 14:38:14.847537 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.848058 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.848870 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.849100 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.849178 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:14.849389 (+ 211us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 14:38:14.850594 (+ 1205us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0928 14:38:14.850737 (+ 143us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42130: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":37}
W20260928 14:38:14.850900 27985 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42130: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260928 14:38:14.851310 28003 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:14.847012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:37851 (local address 127.0.0.1:42130)
0928 14:38:14.847182 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:14.847192 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:14.847257 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:14.847411 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:14.847412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:14.847414 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0928 14:38:14.847470 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:14.847472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.848166 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.848168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:14.848766 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:14.848768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:14.849202 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:14.849203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:14.849263 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:14.849269 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 14:38:14.850711 (+ 1442us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0928 14:38:14.850833 (+ 122us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.13.126:37851: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":105,"threads_started":1}
I20260928 14:38:14.851586 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 27907
E20260928 14:38:14.861559 28004 webserver.cc:533] Webserver: sslize failed:
E20260928 14:38:14.861668 28004 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-28T14:38:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (204 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-28T14:38:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:14Z Disabled control of system clock
I20260928 14:38:14.896945 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:37529
--webserver_interface=127.22.13.126
--webserver_port=0
--builtin_ntp_servers=127.22.13.84:38607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:37529
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/testchainca.cert with env {}
W20260928 14:38:14.978399 28008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:14.978654 28008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:14.978694 28008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:14.979890 28008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:14.979969 28008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:14.979990 28008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:14.980008 28008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:14.980039 28008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 14:38:14.980062 28008 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 14:38:14.980082 28008 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260928 14:38:14.981598 28008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:38607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:37529
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:37529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:14.981901 28008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:14.982151 28008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:14.987067 28016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:14.987035 28013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:14.987041 28014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:14.987435 28008 server_base.cc:1034] running on GCE node
I20260928 14:38:14.987707 28008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:14.988026 28008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:14.989228 28008 hybrid_clock.cc:648] HybridClock initialized: now 1790606294989201 us; error 40 us; skew 500 ppm
I20260928 14:38:14.990680 28008 webserver.cc:460] Webserver started at http://127.22.13.126:46213/ using document root <none> and password file <none>
I20260928 14:38:14.990989 28008 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:14.991076 28008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:14.991240 28008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:14.992381 28008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "06c427be53634eccb9f95e40e243b256"
format_stamp: "Formatted at 2026-09-28 14:38:14 on dist-test-slave-wwwb"
server_key: "afd23ea7e54c8d1482bcffceed0e76c0"
server_key_iv: "eea66e4a8ede8b8df4e23ba820ccd119"
server_key_version: "encryptionkey@0"
I20260928 14:38:14.992774 28008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "06c427be53634eccb9f95e40e243b256"
format_stamp: "Formatted at 2026-09-28 14:38:14 on dist-test-slave-wwwb"
server_key: "afd23ea7e54c8d1482bcffceed0e76c0"
server_key_iv: "eea66e4a8ede8b8df4e23ba820ccd119"
server_key_version: "encryptionkey@0"
I20260928 14:38:14.994181 28008 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260928 14:38:14.994930 28022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:14.995105 28008 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:14.995201 28008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "06c427be53634eccb9f95e40e243b256"
format_stamp: "Formatted at 2026-09-28 14:38:14 on dist-test-slave-wwwb"
server_key: "afd23ea7e54c8d1482bcffceed0e76c0"
server_key_iv: "eea66e4a8ede8b8df4e23ba820ccd119"
server_key_version: "encryptionkey@0"
I20260928 14:38:14.995289 28008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:15.004700 28008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:15.005077 28008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:15.005249 28008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:15.009569 28008 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:37529
I20260928 14:38:15.009603 28074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:37529 every 8 connection(s)
I20260928 14:38:15.010059 28008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:15.011055 28075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:15.011889 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 28008
I20260928 14:38:15.011996 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:15.012185 22581 external_mini_cluster.cc:1428] Setting key 85f8148dcf66a73ea896d5e4c7245cea
I20260928 14:38:15.014266 28075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256: Bootstrap starting.
I20260928 14:38:15.014937 28075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:15.015264 28075 log.cc:824] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:15.016042 28075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256: No bootstrap required, opened a new log
I20260928 14:38:15.017863 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:15.012730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59786 (local address 127.22.13.126:37529)
0928 14:38:15.012939 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:15.012947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:15.012972 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:38:15.013401 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:15.013408 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:15.013472 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:15.013591 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:15.013824 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:15.014823 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:15.015934 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:15.016393 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:15.016484 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:15.017181 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:15.017184 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:15.017185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:15.017356 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:15.017389 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:15.017400 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:15.017464 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:15.017567 (+ 103us) server_negotiation.cc:299] Negotiation successful
0928 14:38:15.017663 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260928 14:38:15.018483 28075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c427be53634eccb9f95e40e243b256" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37529 } }
I20260928 14:38:15.018654 28075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:15.018707 28075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06c427be53634eccb9f95e40e243b256, State: Initialized, Role: FOLLOWER
I20260928 14:38:15.018834 28075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c427be53634eccb9f95e40e243b256" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37529 } }
I20260928 14:38:15.018920 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:38:15.018934 28075 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:15.018971 28075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:15.019016 28075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:15.019719 28075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c427be53634eccb9f95e40e243b256" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37529 } }
I20260928 14:38:15.019845 28075 leader_election.cc:302] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06c427be53634eccb9f95e40e243b256; no voters:
I20260928 14:38:15.020005 28075 leader_election.cc:288] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:15.020042 28080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:15.020155 28080 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [term 1 LEADER]: Becoming Leader. State: Replica: 06c427be53634eccb9f95e40e243b256, State: Running, Role: LEADER
I20260928 14:38:15.020258 28075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:15.020238 28080 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c427be53634eccb9f95e40e243b256" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37529 } }
I20260928 14:38:15.020606 28081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06c427be53634eccb9f95e40e243b256" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c427be53634eccb9f95e40e243b256" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37529 } } }
I20260928 14:38:15.020690 28081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:15.020921 28086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:15.020932 28082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06c427be53634eccb9f95e40e243b256. Latest consensus state: current_term: 1 leader_uuid: "06c427be53634eccb9f95e40e243b256" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06c427be53634eccb9f95e40e243b256" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 37529 } } }
I20260928 14:38:15.021351 28082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256 [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:15.021493 28086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:15.023125 28086 catalog_manager.cc:1347] Generated new cluster ID: 9d15faecff5c4248b166494649500bd9
I20260928 14:38:15.023180 28086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:38:15.038810 28086 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:15.039594 28086 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:15.044891 28086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06c427be53634eccb9f95e40e243b256: Generated new TSK 0
I20260928 14:38:15.045128 28086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:38:15.148124 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:15.056618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59792 (local address 127.22.13.126:37529)
0928 14:38:15.056689 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:15.056692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:15.056777 (+ 85us) server_negotiation.cc:407] Connection header received
0928 14:38:15.056827 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:15.056828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:15.056842 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:15.056924 (+ 82us) server_negotiation.cc:226] Negotiated authn=JWT
0928 14:38:15.057057 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:15.057871 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:15.058673 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:15.059004 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:15.059078 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:15.059253 (+ 175us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 14:38:15.147834 (+ 88581us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0928 14:38:15.148041 (+ 207us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59792: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":44}
W20260928 14:38:15.148203 28078 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59792: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260928 14:38:15.148242 28104 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:15.056531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:37529 (local address 127.0.0.1:59792)
0928 14:38:15.056691 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:15.056710 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:15.056770 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:15.056930 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:15.056930 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:15.056932 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0928 14:38:15.056990 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:15.056992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:15.058001 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:15.058002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:15.058584 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:15.058585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:15.059089 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:15.059090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:15.059134 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:15.059139 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 14:38:15.147997 (+ 88858us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0928 14:38:15.148123 (+ 126us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.13.126:37529: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260928 14:38:15.148514 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 28008
2026-09-28T14:38:15Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (276 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (89 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (89 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:38:15 dist-test-slave-wwwb krb5kdc[28116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:38:15 dist-test-slave-wwwb krb5kdc[28116](info): set up 2 sockets
Sep 28 14:38:15 dist-test-slave-wwwb krb5kdc[28116](info): commencing operation
krb5kdc: starting...
W20260928 14:38:16.370513 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:38:16 dist-test-slave-wwwb krb5kdc[28116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:38:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:38:16.490046 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:40103
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:40103
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:16.572690 28129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:16.572968 28129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:16.573006 28129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:16.574333 28129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:16.574406 28129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:16.574429 28129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:16.574450 28129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:16.574479 28129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:16.576009 28129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:40103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:40103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:16.576327 28129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:16.576573 28129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:16.581538 28134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:16.581538 28137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:16.581595 28129 server_base.cc:1034] running on GCE node
W20260928 14:38:16.581571 28135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:16.582093 28129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:16.582439 28129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:16.583613 28129 hybrid_clock.cc:648] HybridClock initialized: now 1790606296583605 us; error 41 us; skew 500 ppm
Sep 28 14:38:16 dist-test-slave-wwwb krb5kdc[28116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606296, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:16.589620 28129 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:38:16.590183 28129 webserver.cc:460] Webserver started at http://127.22.13.126:35935/ using document root <none> and password file <none>
I20260928 14:38:16.590387 28129 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:16.590427 28129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:16.590521 28129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:16.591568 28129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "8091c206c1ef4a4e8a643d62b54783f8"
format_stamp: "Formatted at 2026-09-28 14:38:16 on dist-test-slave-wwwb"
server_key: "c71417d3394f976517c3a596b3ce93f8"
server_key_iv: "419cd4dbaf72267c15a0881ce8155d2a"
server_key_version: "encryptionkey@0"
I20260928 14:38:16.591861 28129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "8091c206c1ef4a4e8a643d62b54783f8"
format_stamp: "Formatted at 2026-09-28 14:38:16 on dist-test-slave-wwwb"
server_key: "c71417d3394f976517c3a596b3ce93f8"
server_key_iv: "419cd4dbaf72267c15a0881ce8155d2a"
server_key_version: "encryptionkey@0"
I20260928 14:38:16.593146 28129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:16.593874 28144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:16.594074 28129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:16.594172 28129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "8091c206c1ef4a4e8a643d62b54783f8"
format_stamp: "Formatted at 2026-09-28 14:38:16 on dist-test-slave-wwwb"
server_key: "c71417d3394f976517c3a596b3ce93f8"
server_key_iv: "419cd4dbaf72267c15a0881ce8155d2a"
server_key_version: "encryptionkey@0"
I20260928 14:38:16.594264 28129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:16.610060 28129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:16.612005 28129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:16.612175 28129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:16.616472 28129 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:40103
I20260928 14:38:16.616535 28196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:40103 every 8 connection(s)
I20260928 14:38:16.616887 28129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:16.617774 28197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:16.620193 28197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8: Bootstrap starting.
I20260928 14:38:16.620755 28197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:16.620991 28197 log.cc:824] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:16.621680 28197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8: No bootstrap required, opened a new log
I20260928 14:38:16.623396 28197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8091c206c1ef4a4e8a643d62b54783f8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40103 } }
I20260928 14:38:16.623579 28197 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:16.623602 28197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8091c206c1ef4a4e8a643d62b54783f8, State: Initialized, Role: FOLLOWER
I20260928 14:38:16.623723 28197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8091c206c1ef4a4e8a643d62b54783f8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40103 } }
I20260928 14:38:16.623812 28197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:16.623852 28197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:16.623911 28197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:16.624495 28197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8091c206c1ef4a4e8a643d62b54783f8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40103 } }
I20260928 14:38:16.624626 28197 leader_election.cc:302] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8091c206c1ef4a4e8a643d62b54783f8; no voters:
I20260928 14:38:16.624825 28197 leader_election.cc:288] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:16.624879 28200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:16.625058 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 28129
I20260928 14:38:16.625157 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:16.625172 28197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:16.625360 22581 external_mini_cluster.cc:1428] Setting key ed3e3df91365bd4f3de98fbc99e4b9d2
I20260928 14:38:16.625361 28200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [term 1 LEADER]: Becoming Leader. State: Replica: 8091c206c1ef4a4e8a643d62b54783f8, State: Running, Role: LEADER
I20260928 14:38:16.625468 28200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8091c206c1ef4a4e8a643d62b54783f8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40103 } }
I20260928 14:38:16.625809 28201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8091c206c1ef4a4e8a643d62b54783f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8091c206c1ef4a4e8a643d62b54783f8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40103 } } }
I20260928 14:38:16.625890 28201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:16.626147 28202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8091c206c1ef4a4e8a643d62b54783f8. Latest consensus state: current_term: 1 leader_uuid: "8091c206c1ef4a4e8a643d62b54783f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8091c206c1ef4a4e8a643d62b54783f8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40103 } } }
I20260928 14:38:16.626276 28202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8 [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:16.626427 28209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:16.627288 28209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:16.629340 28209 catalog_manager.cc:1347] Generated new cluster ID: 376a8b2df1014adab12d4783b35470b5
I20260928 14:38:16.629405 28209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:38:16 dist-test-slave-wwwb krb5kdc[28116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:16.637784 28208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:16.625898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52128 (local address 127.22.13.126:40103)
0928 14:38:16.626137 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:16.626148 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:16.626184 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:38:16.626313 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:16.626321 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:16.626388 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:16.627428 (+ 1040us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:16.627537 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:16.628785 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:16.629809 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:16.630501 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:16.630618 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:16.633499 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:16.633505 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:16.633506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:16.633540 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:16.636011 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:16.636475 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:16.636478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:16.636478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:16.636601 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:16.636826 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:16.636827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:16.636827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:16.637145 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:16.637294 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:16.637412 (+ 118us) server_negotiation.cc:299] Negotiation successful
0928 14:38:16.637547 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":114,"threads_started":1}
I20260928 14:38:16.639951 28209 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:16.640533 28209 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:16.645814 28209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8091c206c1ef4a4e8a643d62b54783f8: Generated new TSK 0
I20260928 14:38:16.646087 28209 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:38:16.674599 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40103
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:16.787089 28223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:16.787376 28223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:16.787422 28223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:16.788704 28223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:16.788779 28223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:16.788827 28223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:38:16.790627 28223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:16.790906 28223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:16.791150 28223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:16.797106 28229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:16.797135 28231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:16.797477 28223 server_base.cc:1034] running on GCE node
W20260928 14:38:16.797528 28228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:16.797755 28223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:16.798062 28223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:16.799222 28223 hybrid_clock.cc:648] HybridClock initialized: now 1790606296799215 us; error 37 us; skew 500 ppm
Sep 28 14:38:16 dist-test-slave-wwwb krb5kdc[28116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606296, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:16.801834 28223 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:38:16.802380 28223 webserver.cc:460] Webserver started at http://127.22.13.65:38377/ using document root <none> and password file <none>
I20260928 14:38:16.802634 28223 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:16.802716 28223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:16.802870 28223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:16.804075 28223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "e7fe7d823a704e42b48582c59d1a54bf"
format_stamp: "Formatted at 2026-09-28 14:38:16 on dist-test-slave-wwwb"
server_key: "39b8c478844e4dc896e714150b1c642f"
server_key_iv: "2fa4ab01bbb53c5306a9f91e16e3e290"
server_key_version: "encryptionkey@0"
I20260928 14:38:16.804459 28223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "e7fe7d823a704e42b48582c59d1a54bf"
format_stamp: "Formatted at 2026-09-28 14:38:16 on dist-test-slave-wwwb"
server_key: "39b8c478844e4dc896e714150b1c642f"
server_key_iv: "2fa4ab01bbb53c5306a9f91e16e3e290"
server_key_version: "encryptionkey@0"
I20260928 14:38:16.805874 28223 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:38:16.806695 28238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:16.806880 28223 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:38:16.806985 28223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "e7fe7d823a704e42b48582c59d1a54bf"
format_stamp: "Formatted at 2026-09-28 14:38:16 on dist-test-slave-wwwb"
server_key: "39b8c478844e4dc896e714150b1c642f"
server_key_iv: "2fa4ab01bbb53c5306a9f91e16e3e290"
server_key_version: "encryptionkey@0"
I20260928 14:38:16.807075 28223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:16.826632 28223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:16.828665 28223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:16.828883 28223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:16.829141 28223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:16.829466 28223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:16.829521 28223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:16.829564 28223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:16.829591 28223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:16.835760 28223 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:34999
I20260928 14:38:16.835811 28351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:34999 every 8 connection(s)
I20260928 14:38:16.836150 28223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:38:16.840943 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 28223
I20260928 14:38:16.841075 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:38:16.841306 22581 external_mini_cluster.cc:1428] Setting key 1392ee52ae6467e2bccd3e3f21364e05
Sep 28 14:38:16 dist-test-slave-wwwb krb5kdc[28116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606296, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:16.846515 28208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:16.837058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:41539 (local address 127.22.13.126:40103)
0928 14:38:16.837123 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:16.837125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:16.837356 (+ 231us) server_negotiation.cc:407] Connection header received
0928 14:38:16.838371 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:16.838372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:16.838387 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:16.838455 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:16.839069 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:16.839591 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:16.840597 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:16.840908 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:16.840986 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:16.843372 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:16.843374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:16.843374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:16.843383 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:16.844826 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:16.845360 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:16.845361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:16.845361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:16.845418 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:16.845775 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:16.845776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:16.845776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:16.845894 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:16.846012 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:16.846362 (+ 350us) server_negotiation.cc:299] Negotiation successful
0928 14:38:16.846446 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:38:16.846885 28354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:16.837049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:40103 (local address 127.22.13.65:41539)
0928 14:38:16.837218 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:16.837262 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:16.838273 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:16.838517 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:16.838520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:16.838792 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:16.838994 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:16.838999 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:16.839676 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:16.839677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:16.840512 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:16.840514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:16.841001 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:16.841002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:16.841209 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:16.841814 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:16.841822 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:16.843263 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:16.844922 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:16.844931 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:16.844933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:16.845286 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:16.845519 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:16.845521 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:16.845521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:16.845694 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:16.846007 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:16.846015 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:16.846211 (+ 196us) client_negotiation.cc:769] Sending connection context
0928 14:38:16.846345 (+ 134us) client_negotiation.cc:240] Negotiation successful
0928 14:38:16.846560 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260928 14:38:16.847514 28352 heartbeater.cc:347] Connected to a master server at 127.22.13.126:40103
I20260928 14:38:16.847633 28352 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:16.847865 28352 heartbeater.cc:511] Master 127.22.13.126:40103 requested a full tablet report, sending...
I20260928 14:38:16.848426 28161 ts_manager.cc:194] Registered new tserver with Master: e7fe7d823a704e42b48582c59d1a54bf (127.22.13.65:34999)
I20260928 14:38:16.849052 28161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:41539
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:38:16.868466 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40103
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:16.949712 28357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:16.949929 28357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:16.949985 28357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:16.951463 28357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:16.951519 28357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:16.951547 28357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:38:16.953146 28357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:16.953435 28357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:16.953696 28357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:16.958778 28363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:16.958731 28365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:16.958731 28362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:16.959174 28357 server_base.cc:1034] running on GCE node
I20260928 14:38:16.959421 28357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:16.959682 28357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:16.960846 28357 hybrid_clock.cc:648] HybridClock initialized: now 1790606296960843 us; error 36 us; skew 500 ppm
Sep 28 14:38:16 dist-test-slave-wwwb krb5kdc[28116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606296, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:16.963454 28357 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:38:16.963958 28357 webserver.cc:460] Webserver started at http://127.22.13.66:45813/ using document root <none> and password file <none>
I20260928 14:38:16.964164 28357 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:16.964210 28357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:16.964305 28357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:16.965355 28357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "06bc43dec38d4343be0535d1a8dd39da"
format_stamp: "Formatted at 2026-09-28 14:38:16 on dist-test-slave-wwwb"
server_key: "87685c390838f3fe009c44d630eb0dfe"
server_key_iv: "76b429c313ab7522db47e9cc5cc0e913"
server_key_version: "encryptionkey@0"
I20260928 14:38:16.965668 28357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "06bc43dec38d4343be0535d1a8dd39da"
format_stamp: "Formatted at 2026-09-28 14:38:16 on dist-test-slave-wwwb"
server_key: "87685c390838f3fe009c44d630eb0dfe"
server_key_iv: "76b429c313ab7522db47e9cc5cc0e913"
server_key_version: "encryptionkey@0"
I20260928 14:38:16.967083 28357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:38:16.967782 28372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:16.967979 28357 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:38:16.968066 28357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "06bc43dec38d4343be0535d1a8dd39da"
format_stamp: "Formatted at 2026-09-28 14:38:16 on dist-test-slave-wwwb"
server_key: "87685c390838f3fe009c44d630eb0dfe"
server_key_iv: "76b429c313ab7522db47e9cc5cc0e913"
server_key_version: "encryptionkey@0"
I20260928 14:38:16.968125 28357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:16.987089 28357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:16.989065 28357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:16.989180 28357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:16.989370 28357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:16.989704 28357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:16.989748 28357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:16.989784 28357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:16.989830 28357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:16.995772 28357 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:35577
I20260928 14:38:16.995849 28485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:35577 every 8 connection(s)
I20260928 14:38:16.996182 28357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
Sep 28 14:38:16 dist-test-slave-wwwb krb5kdc[28116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606296, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:17.003839 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 28357
I20260928 14:38:17.003964 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:38:17.004223 22581 external_mini_cluster.cc:1428] Setting key ad4276132212d9d42ab66efc1ac127d4
I20260928 14:38:17.006392 28208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:16.996936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:57597 (local address 127.22.13.126:40103)
0928 14:38:16.997003 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:16.997005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:16.997252 (+ 247us) server_negotiation.cc:407] Connection header received
0928 14:38:16.998362 (+ 1110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:16.998364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:16.998377 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:16.998437 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:16.999139 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:16.999663 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:17.000711 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:17.000929 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:17.001023 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:17.003280 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:17.003281 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:17.003282 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:17.003290 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:17.004773 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:17.005250 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:17.005251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:17.005252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:17.005323 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:17.005607 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:17.005608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:17.005609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:17.005744 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:17.005826 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:17.006190 (+ 364us) server_negotiation.cc:299] Negotiation successful
0928 14:38:17.006299 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:38:17.006605 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:16.996939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:40103 (local address 127.22.13.66:57597)
0928 14:38:16.997112 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:16.997150 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:16.998211 (+ 1061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:16.998524 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:16.998527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:16.998815 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:16.999029 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:16.999033 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:16.999779 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:16.999781 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:17.000606 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:17.000609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:17.001035 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:17.001037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:17.001184 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:17.001675 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:17.001684 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:17.003161 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:17.004882 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:17.004891 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:17.004893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:17.005163 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:17.005399 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:17.005400 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:17.005400 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:17.005534 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:17.005816 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:17.005823 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:17.006044 (+ 221us) client_negotiation.cc:769] Sending connection context
0928 14:38:17.006170 (+ 126us) client_negotiation.cc:240] Negotiation successful
0928 14:38:17.006347 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260928 14:38:17.007130 28486 heartbeater.cc:347] Connected to a master server at 127.22.13.126:40103
I20260928 14:38:17.007261 28486 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:17.007472 28486 heartbeater.cc:511] Master 127.22.13.126:40103 requested a full tablet report, sending...
I20260928 14:38:17.007938 28161 ts_manager.cc:194] Registered new tserver with Master: 06bc43dec38d4343be0535d1a8dd39da (127.22.13.66:35577)
I20260928 14:38:17.008481 28161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:57597
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:38:17.029039 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40103
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:17.112495 28491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:17.112725 28491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:17.112749 28491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:17.114090 28491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:17.114140 28491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:17.114167 28491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:38:17.115682 28491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:17.115896 28491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:17.116099 28491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:17.121474 28496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:17.121522 28497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:17.121507 28499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:17.121893 28491 server_base.cc:1034] running on GCE node
I20260928 14:38:17.122129 28491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:17.122339 28491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:17.123482 28491 hybrid_clock.cc:648] HybridClock initialized: now 1790606297123458 us; error 45 us; skew 500 ppm
Sep 28 14:38:17 dist-test-slave-wwwb krb5kdc[28116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606297, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:17.126314 28491 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:38:17.126868 28491 webserver.cc:460] Webserver started at http://127.22.13.67:43815/ using document root <none> and password file <none>
I20260928 14:38:17.127147 28491 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:17.127194 28491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:17.127298 28491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:17.128353 28491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "cf2c5cec655349968f3146526cfd5754"
format_stamp: "Formatted at 2026-09-28 14:38:17 on dist-test-slave-wwwb"
server_key: "48f8dc2614692d54ec5dffbddec33d1b"
server_key_iv: "88aa30385d35c8ee772dbc713757f8f6"
server_key_version: "encryptionkey@0"
I20260928 14:38:17.128672 28491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "cf2c5cec655349968f3146526cfd5754"
format_stamp: "Formatted at 2026-09-28 14:38:17 on dist-test-slave-wwwb"
server_key: "48f8dc2614692d54ec5dffbddec33d1b"
server_key_iv: "88aa30385d35c8ee772dbc713757f8f6"
server_key_version: "encryptionkey@0"
I20260928 14:38:17.130013 28491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:17.130757 28506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:17.130911 28491 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:17.130993 28491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "cf2c5cec655349968f3146526cfd5754"
format_stamp: "Formatted at 2026-09-28 14:38:17 on dist-test-slave-wwwb"
server_key: "48f8dc2614692d54ec5dffbddec33d1b"
server_key_iv: "88aa30385d35c8ee772dbc713757f8f6"
server_key_version: "encryptionkey@0"
I20260928 14:38:17.131053 28491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:17.142412 28491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:17.144346 28491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:17.144464 28491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:17.144659 28491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:17.144989 28491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:17.145035 28491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:17.145079 28491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:17.145107 28491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:17.151117 28491 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:39735
I20260928 14:38:17.151201 28619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:39735 every 8 connection(s)
I20260928 14:38:17.151558 28491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:38:17.154151 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 28491
I20260928 14:38:17.154258 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:38:17.154428 22581 external_mini_cluster.cc:1428] Setting key 62d2f60c3e43077ec677d597f4e91731
Sep 28 14:38:17 dist-test-slave-wwwb krb5kdc[28116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606297, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:17.161527 28208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:17.152396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:38067 (local address 127.22.13.126:40103)
0928 14:38:17.152465 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:17.152467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:17.152667 (+ 200us) server_negotiation.cc:407] Connection header received
0928 14:38:17.153639 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:17.153640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:17.153650 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:17.153711 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:17.154523 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:17.155242 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:17.156319 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:17.156547 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:17.156635 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:17.158888 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:17.158889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:17.158889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:17.158898 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:17.160115 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:17.160536 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:17.160538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:17.160538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:17.160601 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:17.160854 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:17.160855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:17.160855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:17.160982 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:17.161058 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:17.161342 (+ 284us) server_negotiation.cc:299] Negotiation successful
0928 14:38:17.161435 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 14:38:17.161753 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:17.152378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:40103 (local address 127.22.13.67:38067)
0928 14:38:17.152535 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:17.152579 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:17.153540 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:17.153781 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:17.153785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:17.154141 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:17.154418 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:17.154424 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:17.155336 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:17.155338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:17.156219 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:17.156222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:17.156650 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:17.156651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:17.156787 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:17.157329 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:17.157338 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:17.158778 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:17.160219 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:17.160225 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:17.160228 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:17.160456 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:17.160675 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:17.160676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:17.160676 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:17.160788 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:17.161049 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:17.161055 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:17.161209 (+ 154us) client_negotiation.cc:769] Sending connection context
0928 14:38:17.161322 (+ 113us) client_negotiation.cc:240] Negotiation successful
0928 14:38:17.161491 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260928 14:38:17.162232 28620 heartbeater.cc:347] Connected to a master server at 127.22.13.126:40103
I20260928 14:38:17.162336 28620 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:17.162536 28620 heartbeater.cc:511] Master 127.22.13.126:40103 requested a full tablet report, sending...
I20260928 14:38:17.162920 28161 ts_manager.cc:194] Registered new tserver with Master: cf2c5cec655349968f3146526cfd5754 (127.22.13.67:39735)
I20260928 14:38:17.163317 28161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:38067
I20260928 14:38:17.166299 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/test-admin.keytab.
Sep 28 14:38:17 dist-test-slave-wwwb krb5kdc[28116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:17.182261 22581 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260928 14:38:17.184033 28627 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:17.183160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50526 (local address 127.22.13.65:34999)
0928 14:38:17.183395 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:17.183400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:17.183411 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:38:17.183439 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:17.183440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:17.183455 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:17.183526 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:17.183935 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50526: BlockingRecv error: recv got EOF from 127.0.0.1:50526 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":146,"threads_started":1}
W20260928 14:38:17.184111 22581 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/krb5cc' of type FILE
Sep 28 14:38:17 dist-test-slave-wwwb krb5kdc[28116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:17.185122 22581 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 14:38:17 dist-test-slave-wwwb krb5kdc[28116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:17.186082 22581 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 14:38:17 dist-test-slave-wwwb krb5kdc[28116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.65@KRBTEST.COM
I20260928 14:38:17.194303 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:17.186649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50542 (local address 127.22.13.65:34999)
0928 14:38:17.186706 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:17.186708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:17.186767 (+ 59us) server_negotiation.cc:407] Connection header received
0928 14:38:17.186812 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:17.186813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:17.186823 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:17.186872 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:17.187252 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:17.187979 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:17.188711 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:17.189047 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:17.189103 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:17.191421 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:17.191432 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:17.191433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:17.191449 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:17.193013 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:17.193404 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:17.193409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:17.193410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:17.193472 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:17.193679 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:17.193680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:17.193680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:17.193833 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:17.194048 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:17.194093 (+ 45us) server_negotiation.cc:299] Negotiation successful
0928 14:38:17.194171 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260928 14:38:17.196147 28627 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:17.195620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50546 (local address 127.22.13.65:34999)
0928 14:38:17.195656 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:17.195658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:17.195693 (+ 35us) server_negotiation.cc:407] Connection header received
0928 14:38:17.195752 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:17.195753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:17.195762 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:17.195815 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:17.196054 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50546: BlockingRecv error: recv got EOF from 127.0.0.1:50546 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260928 14:38:17.196313 22581 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/krb5cc)
Sep 28 14:38:17 dist-test-slave-wwwb krb5kdc[28116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:17.197458 22581 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 14:38:17 dist-test-slave-wwwb krb5kdc[28116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:17.198488 22581 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 14:38:17 dist-test-slave-wwwb krb5kdc[28116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.65@KRBTEST.COM
I20260928 14:38:17.205340 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:17.199001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50550 (local address 127.22.13.65:34999)
0928 14:38:17.199062 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:17.199064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:17.199117 (+ 53us) server_negotiation.cc:407] Connection header received
0928 14:38:17.199203 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:17.199204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:17.199214 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:17.199285 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:17.199662 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:17.200166 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:17.200897 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:17.201103 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:17.201189 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:17.203250 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:17.203251 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:17.203251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:17.203258 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:17.204424 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:17.204739 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:17.204740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:17.204740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:17.204790 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:17.204951 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:17.204952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:17.204952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:17.205041 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:17.205113 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:17.205217 (+ 104us) server_negotiation.cc:299] Negotiation successful
0928 14:38:17.205278 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260928 14:38:17.207047 28627 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:17.206425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50562 (local address 127.22.13.65:34999)
0928 14:38:17.206463 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:17.206465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:17.206538 (+ 73us) server_negotiation.cc:407] Connection header received
0928 14:38:17.206610 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:17.206611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:17.206620 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:17.206690 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:17.206991 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50562: BlockingRecv error: recv got EOF from 127.0.0.1:50562 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260928 14:38:17.207160 22581 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606270465379-22581-0/krb5kdc/krb5cc' of type FILE
Sep 28 14:38:17 dist-test-slave-wwwb krb5kdc[28116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:17.208118 22581 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 14:38:17 dist-test-slave-wwwb krb5kdc[28116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:17.208942 22581 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 14:38:17 dist-test-slave-wwwb krb5kdc[28116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.65@KRBTEST.COM
I20260928 14:38:17.216094 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:17.209488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50578 (local address 127.22.13.65:34999)
0928 14:38:17.209570 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:17.209572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:17.209648 (+ 76us) server_negotiation.cc:407] Connection header received
0928 14:38:17.209735 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:17.209736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:17.209745 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:17.209817 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:17.210254 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:17.210718 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:17.211407 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:17.211603 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:17.211713 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:17.213716 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:17.213717 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:17.213718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:17.213725 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:17.214962 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:17.215322 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:17.215324 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:17.215324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:17.215383 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:17.215579 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:17.215580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:17.215580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:17.215690 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:17.215804 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:17.215911 (+ 107us) server_negotiation.cc:299] Negotiation successful
0928 14:38:17.215990 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 14:38:17.216914 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 28223
I20260928 14:38:17.221477 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 28357
I20260928 14:38:17.226119 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 28491
I20260928 14:38:17.230717 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 28129
2026-09-28T14:38:17Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1904 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:38:17 dist-test-slave-wwwb krb5kdc[28640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:38:17 dist-test-slave-wwwb krb5kdc[28640](info): set up 2 sockets
Sep 28 14:38:17 dist-test-slave-wwwb krb5kdc[28640](info): commencing operation
krb5kdc: starting...
W20260928 14:38:18.262425 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:38:18 dist-test-slave-wwwb krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:38:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:18Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.126.keytab.
Entry for principal oryx/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.126.keytab.
I20260928 14:38:18.378366 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:36117
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.126.keytab
--principal=oryx/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:36117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:18.460739 28653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:18.461010 28653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:18.461071 28653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:18.462338 28653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:18.462407 28653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:18.462430 28653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:18.462450 28653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:18.462479 28653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:18.464057 28653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:36117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.126.keytab
--principal=oryx/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:36117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:18.464365 28653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:18.464610 28653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:18.469686 28658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:18.469673 28659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:18.470003 28653 server_base.cc:1034] running on GCE node
W20260928 14:38:18.469673 28661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:18.470335 28653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:18.470662 28653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:18.471835 28653 hybrid_clock.cc:648] HybridClock initialized: now 1790606298471821 us; error 44 us; skew 500 ppm
Sep 28 14:38:18 dist-test-slave-wwwb krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606298, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:18.474442 28653 init.cc:377] Logged in from keytab as oryx/127.22.13.126@KRBTEST.COM (short username oryx)
I20260928 14:38:18.474931 28653 webserver.cc:460] Webserver started at http://127.22.13.126:45431/ using document root <none> and password file <none>
I20260928 14:38:18.475180 28653 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:18.475255 28653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:18.475430 28653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:18.476564 28653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "8c3817afa240422eb13936fc7717656b"
format_stamp: "Formatted at 2026-09-28 14:38:18 on dist-test-slave-wwwb"
server_key: "7ccb340f03ac1a3f1d144418bb7d5778"
server_key_iv: "8fc8a7e18682cc0a8fe04eac5ac34b16"
server_key_version: "encryptionkey@0"
I20260928 14:38:18.477013 28653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "8c3817afa240422eb13936fc7717656b"
format_stamp: "Formatted at 2026-09-28 14:38:18 on dist-test-slave-wwwb"
server_key: "7ccb340f03ac1a3f1d144418bb7d5778"
server_key_iv: "8fc8a7e18682cc0a8fe04eac5ac34b16"
server_key_version: "encryptionkey@0"
I20260928 14:38:18.479074 28653 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260928 14:38:18.480005 28668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:18.480243 28653 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:18.480341 28653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "8c3817afa240422eb13936fc7717656b"
format_stamp: "Formatted at 2026-09-28 14:38:18 on dist-test-slave-wwwb"
server_key: "7ccb340f03ac1a3f1d144418bb7d5778"
server_key_iv: "8fc8a7e18682cc0a8fe04eac5ac34b16"
server_key_version: "encryptionkey@0"
I20260928 14:38:18.480422 28653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:18.491235 28653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:18.493175 28653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:18.493374 28653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:18.497682 28653 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:36117
I20260928 14:38:18.497722 28720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:36117 every 8 connection(s)
I20260928 14:38:18.498157 28653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:18.499311 28721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:18.501659 28721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b: Bootstrap starting.
I20260928 14:38:18.502321 28721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:18.502573 28721 log.cc:824] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:18.503337 28721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b: No bootstrap required, opened a new log
I20260928 14:38:18.503592 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 28653
I20260928 14:38:18.503719 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:18.503930 22581 external_mini_cluster.cc:1428] Setting key 56e11e2529863015373e6e3291577d52
I20260928 14:38:18.505736 28721 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } }
I20260928 14:38:18.505996 28721 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:18.506054 28721 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3817afa240422eb13936fc7717656b, State: Initialized, Role: FOLLOWER
I20260928 14:38:18.506213 28721 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } }
I20260928 14:38:18.506317 28721 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:18.506390 28721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:18.506444 28721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:18.507087 28721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } }
I20260928 14:38:18.507177 28721 leader_election.cc:302] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c3817afa240422eb13936fc7717656b; no voters:
I20260928 14:38:18.507409 28721 leader_election.cc:288] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:18.507432 28726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:18.507537 28726 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [term 1 LEADER]: Becoming Leader. State: Replica: 8c3817afa240422eb13936fc7717656b, State: Running, Role: LEADER
I20260928 14:38:18.507619 28726 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } }
I20260928 14:38:18.507686 28721 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:18.507974 28726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c3817afa240422eb13936fc7717656b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } }
I20260928 14:38:18.508045 28726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:18.508180 28726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c3817afa240422eb13936fc7717656b. Latest consensus state: current_term: 1 leader_uuid: "8c3817afa240422eb13936fc7717656b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } }
I20260928 14:38:18.508247 28726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:18.508380 28731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:18.508929 28731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:18.511075 28731 catalog_manager.cc:1347] Generated new cluster ID: a0d4d515ab95487f91125cdd920f6af3
I20260928 14:38:18.511132 28731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:38:18 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM
I20260928 14:38:18.516193 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:18.504508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46152 (local address 127.22.13.126:36117)
0928 14:38:18.504723 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:18.504732 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:18.504760 (+ 28us) server_negotiation.cc:407] Connection header received
0928 14:38:18.505248 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:18.505255 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:18.505308 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:18.505442 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:18.506374 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:18.507956 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:18.509199 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:18.509777 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:18.509890 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:18.512610 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:18.512614 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:18.512614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:18.512628 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:18.514500 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:18.514908 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:18.514912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:18.514913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:18.515009 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:18.515225 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:18.515226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:18.515226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:18.515493 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:18.515688 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:18.515772 (+ 84us) server_negotiation.cc:299] Negotiation successful
0928 14:38:18.516026 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":113,"threads_started":1}
I20260928 14:38:18.531824 28731 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:18.532395 28731 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:18.536372 28731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b: Generated new TSK 0
I20260928 14:38:18.536588 28731 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.65.keytab.
Entry for principal oryx/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.65.keytab.
I20260928 14:38:18.568281 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36117
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.65.keytab
--principal=oryx/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:18.651997 28747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:18.652236 28747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:18.652262 28747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:18.653565 28747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:18.653620 28747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:18.653648 28747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:38:18.655340 28747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.65.keytab
--principal=oryx/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:18.655562 28747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:18.655764 28747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:18.660885 28753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:18.661152 28747 server_base.cc:1034] running on GCE node
W20260928 14:38:18.661216 28752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:18.661449 28755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:18.661721 28747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:18.661998 28747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:18.663141 28747 hybrid_clock.cc:648] HybridClock initialized: now 1790606298663126 us; error 31 us; skew 500 ppm
Sep 28 14:38:18 dist-test-slave-wwwb krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606298, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:18.665798 28747 init.cc:377] Logged in from keytab as oryx/127.22.13.65@KRBTEST.COM (short username oryx)
I20260928 14:38:18.666404 28747 webserver.cc:460] Webserver started at http://127.22.13.65:39501/ using document root <none> and password file <none>
I20260928 14:38:18.666662 28747 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:18.666750 28747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:18.666903 28747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:18.668004 28747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "456ec803412d4816b65345ebf7a8d8a5"
format_stamp: "Formatted at 2026-09-28 14:38:18 on dist-test-slave-wwwb"
server_key: "75c78655e475dc387a77e5d849c9610b"
server_key_iv: "83a1554d502c32218cde2d2b7bd1bdaf"
server_key_version: "encryptionkey@0"
I20260928 14:38:18.668391 28747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "456ec803412d4816b65345ebf7a8d8a5"
format_stamp: "Formatted at 2026-09-28 14:38:18 on dist-test-slave-wwwb"
server_key: "75c78655e475dc387a77e5d849c9610b"
server_key_iv: "83a1554d502c32218cde2d2b7bd1bdaf"
server_key_version: "encryptionkey@0"
I20260928 14:38:18.669791 28747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:38:18.670707 28762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:18.670905 28747 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:18.671007 28747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "456ec803412d4816b65345ebf7a8d8a5"
format_stamp: "Formatted at 2026-09-28 14:38:18 on dist-test-slave-wwwb"
server_key: "75c78655e475dc387a77e5d849c9610b"
server_key_iv: "83a1554d502c32218cde2d2b7bd1bdaf"
server_key_version: "encryptionkey@0"
I20260928 14:38:18.671098 28747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:18.693029 28747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:18.695603 28747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:18.695804 28747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:18.696051 28747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:18.696391 28747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:18.696444 28747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:18.696487 28747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:18.696516 28747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:18.702422 28747 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:41419
I20260928 14:38:18.702468 28875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:41419 every 8 connection(s)
I20260928 14:38:18.702821 28747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:38:18.703630 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 28747
I20260928 14:38:18.703784 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:38:18.704001 22581 external_mini_cluster.cc:1428] Setting key 5fedac7fce5ff612505dcff263e34b21
Sep 28 14:38:18 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606298, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.22.13.66@KRBTEST.COM; defaulting to no policy
I20260928 14:38:18.714260 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:18.703606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:43309 (local address 127.22.13.126:36117)
0928 14:38:18.703651 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:18.703657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:18.703988 (+ 331us) server_negotiation.cc:407] Connection header received
0928 14:38:18.704630 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:18.704631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:18.704640 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:18.704680 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:18.705713 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:18.706424 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:18.707817 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:18.708173 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:18.708265 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:18.711071 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:18.711073 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:18.711074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:18.711084 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:18.712730 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:18.713215 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:18.713217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:18.713217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:18.713282 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:18.713560 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:18.713560 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:18.713561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:18.713680 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:18.713737 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:18.714054 (+ 317us) server_negotiation.cc:299] Negotiation successful
0928 14:38:18.714153 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 14:38:18.714462 28878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:18.703618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.65:43309)
0928 14:38:18.703830 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:18.703882 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:18.704522 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:18.704764 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:18.704767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:18.705367 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:18.705590 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:18.705595 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:18.706527 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:18.706529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:18.707660 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:18.707664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:18.708285 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:18.708286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:18.708438 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:18.709113 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:18.709122 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:18.710955 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:18.712823 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:18.712831 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:18.712833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:18.713132 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:18.713383 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:18.713383 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:18.713384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:18.713498 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:18.713741 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:18.713746 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:18.713879 (+ 133us) client_negotiation.cc:769] Sending connection context
0928 14:38:18.714028 (+ 149us) client_negotiation.cc:240] Negotiation successful
0928 14:38:18.714211 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":118,"threads_started":1}
I20260928 14:38:18.715103 28876 heartbeater.cc:347] Connected to a master server at 127.22.13.126:36117
I20260928 14:38:18.715216 28876 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.13.66@KRBTEST.COM" created.
I20260928 14:38:18.715425 28876 heartbeater.cc:511] Master 127.22.13.126:36117 requested a full tablet report, sending...
I20260928 14:38:18.716025 28685 ts_manager.cc:194] Registered new tserver with Master: 456ec803412d4816b65345ebf7a8d8a5 (127.22.13.65:41419)
I20260928 14:38:18.716706 28685 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:43309
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.66.keytab.
Entry for principal oryx/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.66.keytab.
I20260928 14:38:18.728907 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36117
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.66.keytab
--principal=oryx/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:18.811705 28881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:18.811936 28881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:18.811964 28881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:18.813098 28881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:18.813138 28881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:18.813165 28881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:38:18.814662 28881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.66.keytab
--principal=oryx/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:18.814884 28881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:18.815083 28881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:18.820135 28887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:18.820096 28886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:18.820050 28889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:18.820452 28881 server_base.cc:1034] running on GCE node
I20260928 14:38:18.820670 28881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:18.820902 28881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:18.822026 28881 hybrid_clock.cc:648] HybridClock initialized: now 1790606298822017 us; error 35 us; skew 500 ppm
Sep 28 14:38:18 dist-test-slave-wwwb krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606298, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:18.824669 28881 init.cc:377] Logged in from keytab as oryx/127.22.13.66@KRBTEST.COM (short username oryx)
I20260928 14:38:18.825151 28881 webserver.cc:460] Webserver started at http://127.22.13.66:39631/ using document root <none> and password file <none>
I20260928 14:38:18.825403 28881 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:18.825474 28881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:18.825623 28881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:18.826784 28881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "d844e25249cd4df5a06f0884bf9850f5"
format_stamp: "Formatted at 2026-09-28 14:38:18 on dist-test-slave-wwwb"
server_key: "51c5baba3ba5b84408efeafe5f9ff815"
server_key_iv: "902f081161520b6adc58a2cfd154e774"
server_key_version: "encryptionkey@0"
I20260928 14:38:18.827154 28881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "d844e25249cd4df5a06f0884bf9850f5"
format_stamp: "Formatted at 2026-09-28 14:38:18 on dist-test-slave-wwwb"
server_key: "51c5baba3ba5b84408efeafe5f9ff815"
server_key_iv: "902f081161520b6adc58a2cfd154e774"
server_key_version: "encryptionkey@0"
I20260928 14:38:18.828522 28881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:38:18.829249 28896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:18.829419 28881 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:38:18.829526 28881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "d844e25249cd4df5a06f0884bf9850f5"
format_stamp: "Formatted at 2026-09-28 14:38:18 on dist-test-slave-wwwb"
server_key: "51c5baba3ba5b84408efeafe5f9ff815"
server_key_iv: "902f081161520b6adc58a2cfd154e774"
server_key_version: "encryptionkey@0"
I20260928 14:38:18.829607 28881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:18.841833 28881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:18.844383 28881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:18.844555 28881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:18.844799 28881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:18.845116 28881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:18.845170 28881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:18.845206 28881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:18.845253 28881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:18.851209 28881 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:36965
I20260928 14:38:18.851290 29009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:36965 every 8 connection(s)
I20260928 14:38:18.851645 28881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:38:18.854151 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 28881
I20260928 14:38:18.854271 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:38:18.854487 22581 external_mini_cluster.cc:1428] Setting key 7bef9090118f926e22c5c0d475b5d23f
Sep 28 14:38:18 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606298, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM
I20260928 14:38:18.862473 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:18.852459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:53115 (local address 127.22.13.126:36117)
0928 14:38:18.852539 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:18.852541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:18.852738 (+ 197us) server_negotiation.cc:407] Connection header received
0928 14:38:18.853292 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:18.853294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:18.853307 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:18.853348 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:18.854600 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:18.855211 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:18.856272 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:18.856594 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:18.856689 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:18.859219 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:18.859221 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:18.859221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:18.859232 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:18.860948 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:18.861367 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:18.861369 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:18.861369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:18.861439 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:18.861674 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:18.861675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:18.861675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:18.861835 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:18.861930 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:18.862249 (+ 319us) server_negotiation.cc:299] Negotiation successful
0928 14:38:18.862352 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:38:18.862510 29012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:18.852429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.66:53115)
0928 14:38:18.852607 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:18.852645 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:18.853176 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:18.853441 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:18.853445 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:18.854163 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:18.854482 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:18.854489 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:18.855322 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:18.855323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:18.856163 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:18.856166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:18.856706 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:18.856707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:18.856888 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:18.857547 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:18.857559 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:18.859103 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:18.861039 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:18.861045 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:18.861048 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:18.861288 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:18.861510 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:18.861511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:18.861511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:18.861610 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:18.861922 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:18.861926 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:18.862102 (+ 176us) client_negotiation.cc:769] Sending connection context
0928 14:38:18.862199 (+ 97us) client_negotiation.cc:240] Negotiation successful
0928 14:38:18.862333 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260928 14:38:18.863063 29010 heartbeater.cc:347] Connected to a master server at 127.22.13.126:36117
I20260928 14:38:18.863189 29010 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:18.863400 29010 heartbeater.cc:511] Master 127.22.13.126:36117 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.22.13.67@KRBTEST.COM; defaulting to no policy
I20260928 14:38:18.863794 28685 ts_manager.cc:194] Registered new tserver with Master: d844e25249cd4df5a06f0884bf9850f5 (127.22.13.66:36965)
I20260928 14:38:18.864192 28685 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:53115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.67.keytab.
Entry for principal oryx/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.67.keytab.
I20260928 14:38:18.880803 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36117
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.67.keytab
--principal=oryx/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:18.964609 29015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:18.964861 29015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:18.964888 29015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:18.966192 29015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:18.966239 29015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:18.966270 29015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:38:18.967821 29015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.67.keytab
--principal=oryx/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:18.968048 29015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:18.968246 29015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:18.973367 29021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:18.973395 29023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:18.973367 29020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:18.973537 29015 server_base.cc:1034] running on GCE node
I20260928 14:38:18.973803 29015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:18.974155 29015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:18.975332 29015 hybrid_clock.cc:648] HybridClock initialized: now 1790606298975328 us; error 59 us; skew 500 ppm
Sep 28 14:38:18 dist-test-slave-wwwb krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606298, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:18.978026 29015 init.cc:377] Logged in from keytab as oryx/127.22.13.67@KRBTEST.COM (short username oryx)
I20260928 14:38:18.978466 29015 webserver.cc:460] Webserver started at http://127.22.13.67:43743/ using document root <none> and password file <none>
I20260928 14:38:18.978663 29015 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:18.978708 29015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:18.978804 29015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:18.979866 29015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "ac34180e0f1c455da81fb0c4bc578d4b"
format_stamp: "Formatted at 2026-09-28 14:38:18 on dist-test-slave-wwwb"
server_key: "82add26c79ef89108a7fe2920644c2c2"
server_key_iv: "602316d083025aaaad7618e74d3d19af"
server_key_version: "encryptionkey@0"
I20260928 14:38:18.980283 29015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "ac34180e0f1c455da81fb0c4bc578d4b"
format_stamp: "Formatted at 2026-09-28 14:38:18 on dist-test-slave-wwwb"
server_key: "82add26c79ef89108a7fe2920644c2c2"
server_key_iv: "602316d083025aaaad7618e74d3d19af"
server_key_version: "encryptionkey@0"
I20260928 14:38:18.981634 29015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:18.982465 29030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:18.982621 29015 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:18.982710 29015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "ac34180e0f1c455da81fb0c4bc578d4b"
format_stamp: "Formatted at 2026-09-28 14:38:18 on dist-test-slave-wwwb"
server_key: "82add26c79ef89108a7fe2920644c2c2"
server_key_iv: "602316d083025aaaad7618e74d3d19af"
server_key_version: "encryptionkey@0"
I20260928 14:38:18.982802 29015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:18.996865 29015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:18.998951 29015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:18.999130 29015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:18.999374 29015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:18.999718 29015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:18.999752 29015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:18.999778 29015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:18.999838 29015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.005581 29015 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:38887
I20260928 14:38:19.005654 29143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:38887 every 8 connection(s)
I20260928 14:38:19.006102 29015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:38:19.006237 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 29015
I20260928 14:38:19.006361 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:38:19.006608 22581 external_mini_cluster.cc:1428] Setting key a887f84653c5a33aa055c8b82c6ee8e8
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606298, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM
I20260928 14:38:19.016068 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.007094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:41451 (local address 127.22.13.126:36117)
0928 14:38:19.007164 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.007167 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.007461 (+ 294us) server_negotiation.cc:407] Connection header received
0928 14:38:19.008081 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.008082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.008092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.008160 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.009314 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.009863 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.010868 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.011085 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.011142 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.013414 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.013415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.013416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.013423 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.014818 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.015262 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.015263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.015263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.015313 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.015525 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.015526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.015526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.015632 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.015680 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.015934 (+ 254us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.016003 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:38:19.016454 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.007088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.67:41451)
0928 14:38:19.007263 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.007363 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.007987 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.008258 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.008263 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.008948 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.009160 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.009221 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.009972 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.009972 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.010783 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.010786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.011163 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.011164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.011286 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.011778 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.011786 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.013299 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.014898 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.014905 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.014907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.015181 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.015370 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.015371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.015371 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.015473 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.015684 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.015688 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.015826 (+ 138us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.015911 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.016172 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260928 14:38:19.017009 29144 heartbeater.cc:347] Connected to a master server at 127.22.13.126:36117
I20260928 14:38:19.017110 29144 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:19.017320 29144 heartbeater.cc:511] Master 127.22.13.126:36117 requested a full tablet report, sending...
I20260928 14:38:19.017763 28685 ts_manager.cc:194] Registered new tserver with Master: ac34180e0f1c455da81fb0c4bc578d4b (127.22.13.67:38887)
I20260928 14:38:19.018234 28685 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:41451
I20260928 14:38:19.018765 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.13.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.13.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.13.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.125.keytab.
Entry for principal oryx/127.22.13.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.125.keytab.
I20260928 14:38:19.042827 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.125:43783
--webserver_interface=127.22.13.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.125.keytab
--principal=oryx/127.22.13.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:36117,127.22.13.125:43783
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:19.125586 29149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:19.125808 29149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:19.125833 29149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:19.127112 29149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:19.127159 29149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:19.127174 29149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:19.127187 29149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:19.127198 29149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:19.128762 29149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:36117,127.22.13.125:43783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.125.keytab
--principal=oryx/127.22.13.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.125:43783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.13.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
W20260928 14:38:19.128892 29149 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.22.13.126:36117,127.22.13.125:43783'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260928 14:38:19.129065 29149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:19.129310 29149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:19.134282 29155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:19.134282 29157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:19.134279 29154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:19.134527 29149 server_base.cc:1034] running on GCE node
I20260928 14:38:19.134756 29149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:19.135048 29149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:19.136193 29149 hybrid_clock.cc:648] HybridClock initialized: now 1790606299136181 us; error 26 us; skew 500 ppm
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:19.138661 29149 init.cc:377] Logged in from keytab as oryx/127.22.13.125@KRBTEST.COM (short username oryx)
I20260928 14:38:19.139118 29149 webserver.cc:460] Webserver started at http://127.22.13.125:38043/ using document root <none> and password file <none>
I20260928 14:38:19.139369 29149 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:19.139439 29149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:19.139590 29149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:19.140697 29149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/data/instance:
uuid: "ad5654dbe1244b828d6054a73f683c75"
format_stamp: "Formatted at 2026-09-28 14:38:19 on dist-test-slave-wwwb"
server_key: "b22756afebdbdc7ff104386043f3f1e3"
server_key_iv: "41a7fad7e56d35bf259abac286eae60d"
server_key_version: "encryptionkey@0"
I20260928 14:38:19.141070 29149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/wal/instance:
uuid: "ad5654dbe1244b828d6054a73f683c75"
format_stamp: "Formatted at 2026-09-28 14:38:19 on dist-test-slave-wwwb"
server_key: "b22756afebdbdc7ff104386043f3f1e3"
server_key_iv: "41a7fad7e56d35bf259abac286eae60d"
server_key_version: "encryptionkey@0"
I20260928 14:38:19.142542 29149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:38:19.143311 29164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.143498 29149 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:38:19.143605 29149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/wal
uuid: "ad5654dbe1244b828d6054a73f683c75"
format_stamp: "Formatted at 2026-09-28 14:38:19 on dist-test-slave-wwwb"
server_key: "b22756afebdbdc7ff104386043f3f1e3"
server_key_iv: "41a7fad7e56d35bf259abac286eae60d"
server_key_version: "encryptionkey@0"
I20260928 14:38:19.143718 29149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:19.156185 29149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:19.158313 29149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:19.158489 29149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:19.162317 29149 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.125:43783
I20260928 14:38:19.162345 29216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.125:43783 every 8 connection(s)
I20260928 14:38:19.162695 29149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/data/info.pb
I20260928 14:38:19.163691 29217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:19.165268 29217 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:38:19.167999 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 29149
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.125@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.13.125@KRBTEST.COM
I20260928 14:38:19.176265 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.166016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46166 (local address 127.22.13.126:36117)
0928 14:38:19.166115 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.166117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.166302 (+ 185us) server_negotiation.cc:407] Connection header received
0928 14:38:19.166834 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.166835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.166847 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.166904 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.167524 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.168257 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.169955 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.170169 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.170250 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.172855 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.172856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.172857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.172864 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.174487 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.175073 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.175074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.175075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.175139 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.175453 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.175454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.175454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.175686 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.175769 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.176053 (+ 284us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.176152 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 14:38:19.176328 29219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.165987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:46166)
0928 14:38:19.166180 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.166221 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.166705 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.166974 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.166978 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.167237 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.167428 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.167432 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.168343 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.168344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.169847 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.169850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.170266 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.170267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.170408 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.170931 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.170941 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.172760 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.174569 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.174576 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.174578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.174990 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.175233 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.175234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.175234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.175380 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.175765 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.175769 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.175895 (+ 126us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.175983 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.176133 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":96,"threads_started":1}
I20260928 14:38:19.176996 29217 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:38:19.177582 29221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.168516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44296 (local address 127.22.13.125:43783)
0928 14:38:19.168698 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.168702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.168718 (+ 16us) server_negotiation.cc:407] Connection header received
0928 14:38:19.168756 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.168757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.168773 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.168828 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.169553 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.170273 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.171196 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.171482 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.171540 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.174262 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.174270 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.174270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.174277 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.176264 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.176676 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.176679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.176679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.176747 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.176956 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.176956 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.176957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.177120 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.177262 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.177372 (+ 110us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.177461 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.125@KRBTEST.COM for oryx/127.22.13.125@KRBTEST.COM
I20260928 14:38:19.183732 29219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.177257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:43783 (local address 127.0.0.1:44298)
0928 14:38:19.177300 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.177309 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.177341 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.177570 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.177571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.177710 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.177756 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.177758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.178621 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.178621 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.179125 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.179127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.179464 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.179464 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.179513 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.179912 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.179917 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.181006 (+ 1089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.182886 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.182887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.182887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.183070 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.183241 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.183241 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.183242 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.183291 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.183463 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.183463 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.183498 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.183546 (+ 48us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.183619 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260928 14:38:19.183897 29222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.177314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44298 (local address 127.22.13.125:43783)
0928 14:38:19.177447 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.177449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.177454 (+ 5us) server_negotiation.cc:407] Connection header received
0928 14:38:19.177469 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.177469 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.177477 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.177538 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.178006 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.178554 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.179195 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.179402 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.179452 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.181079 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.181080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.181081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.181087 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.182779 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.183133 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.183133 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.183134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.183184 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.183328 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.183328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.183329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.183416 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.183460 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.183559 (+ 99us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.183666 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260928 14:38:19.185143 29217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: Bootstrap starting.
I20260928 14:38:19.185732 29217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:19.186023 29217 log.cc:824] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:19.186723 29217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: No bootstrap required, opened a new log
I20260928 14:38:19.188149 29217 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } }
I20260928 14:38:19.188279 29217 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:19.188301 29217 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5654dbe1244b828d6054a73f683c75, State: Initialized, Role: FOLLOWER
I20260928 14:38:19.188429 29217 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } }
I20260928 14:38:19.188710 29217 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:19.188673 29224 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } } }
I20260928 14:38:19.188782 29224 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: This master's current role is: FOLLOWER
W20260928 14:38:19.190480 29235 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:38:19.190539 29235 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:38:19.192082 29149 master_runner.cc:428] Detected that this master ad5654dbe1244b828d6054a73f683c75 is joining an existing cluster
I20260928 14:38:19.192142 29149 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.13.125:43783
I20260928 14:38:19.199954 29242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.193073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:43783 (local address 127.0.0.1:44302)
0928 14:38:19.193445 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.193455 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.193505 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.193662 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.193662 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.193870 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.193919 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.193921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.194878 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.194879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.195644 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.195645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.196133 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.196134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.196195 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.196545 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.196550 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.196915 (+ 365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.198657 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.198659 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.198659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.198828 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.199036 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.199037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.199037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.199104 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.199516 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.199517 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.199560 (+ 43us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.199627 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.199759 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":69,"threads_started":1}
I20260928 14:38:19.200099 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.193235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46168 (local address 127.22.13.126:36117)
0928 14:38:19.193277 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.193279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.193288 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:38:19.193310 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.193311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.193320 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.193381 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.193831 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.194456 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.195381 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.195693 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.195733 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.197037 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.197038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.197038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.197046 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.198834 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.199231 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.199232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.199232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.199284 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.199510 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.199510 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.199511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.199634 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.199700 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.199911 (+ 211us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.199985 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:38:19.200389 29241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.193028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:46168)
0928 14:38:19.193177 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.193189 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.193243 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.193384 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.193385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.193609 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.193734 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.193736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.194539 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.194539 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.195285 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.195287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.195833 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.195834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.195873 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.196360 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.196365 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.196933 (+ 568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.198926 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.198927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.198928 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.199133 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.199372 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.199373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.199373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.199440 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.199726 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.199727 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.199780 (+ 53us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.200184 (+ 404us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.200299 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260928 14:38:19.199965 29222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.193213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44302 (local address 127.22.13.125:43783)
0928 14:38:19.193251 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.193253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.193518 (+ 265us) server_negotiation.cc:407] Connection header received
0928 14:38:19.193565 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.193566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.193576 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.193637 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.194039 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.194790 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.195739 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.196055 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.196133 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.197019 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.197020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.197021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.197027 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.198559 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.198909 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.198910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.198911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.198961 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.199164 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.199165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.199166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.199290 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.199372 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.199668 (+ 296us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.199860 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":21}
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606298, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.126@KRBTEST.COM for oryx/127.22.13.125@KRBTEST.COM
I20260928 14:38:19.210376 29222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.202787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44314 (local address 127.22.13.125:43783)
0928 14:38:19.202839 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.202841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.203006 (+ 165us) server_negotiation.cc:407] Connection header received
0928 14:38:19.203065 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.203066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.203075 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.203180 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.203662 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.204432 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.205196 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.205498 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.205569 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.207723 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.207724 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.207725 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.207731 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.209071 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.209497 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.209498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.209498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.209563 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.209768 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.209769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.209769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.209883 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.209958 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.210187 (+ 229us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.210270 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 14:38:19.210392 29243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.202725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:43783 (local address 127.0.0.1:44314)
0928 14:38:19.202894 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.202935 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.203005 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.203189 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.203191 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.203496 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.203577 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.203580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.204514 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.204514 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.205115 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.205116 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.205573 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.205574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.205651 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.206168 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.206172 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.207608 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.209158 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.209167 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.209170 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.209404 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.209633 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.209634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.209634 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.209696 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.209976 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.209983 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.210048 (+ 65us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.210147 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.210275 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":84,"threads_started":1}
I20260928 14:38:19.211014 28685 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.13.125:43783: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ad5654dbe1244b828d6054a73f683c75"
member_type: NON_VOTER
last_known_addr {
host: "127.22.13.125"
port: 43783
}
attrs {
promote: true
}
}
dest_uuid: "8c3817afa240422eb13936fc7717656b"
cas_config_opid_index: -1
I20260928 14:38:19.211354 28685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: NON_VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: true } }
I20260928 14:38:19.211860 29245 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ad5654dbe1244b828d6054a73f683c75 (127.22.13.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: NON_VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: true } } }
I20260928 14:38:19.212186 29191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:19.212927 29245 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.13.125:43783
I20260928 14:38:19.213014 29191 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 FOLLOWER]: Refusing update from remote peer 8c3817afa240422eb13936fc7717656b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260928 14:38:19.213186 29149 master.cc:537] Master@127.22.13.125:43783 shutting down...
I20260928 14:38:19.213583 29244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8c3817afa240422eb13936fc7717656b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: NON_VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: true } } }
I20260928 14:38:19.213675 29244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:19.213786 29245 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: NON_VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 14:38:19.214545 28669 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260928 14:38:19.214982 28669 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b -> Peer ad5654dbe1244b828d6054a73f683c75 (127.22.13.125:43783): Couldn't send request to peer ad5654dbe1244b828d6054a73f683c75. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 14:38:19.215926 29149 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:19.216028 29149 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:19.216086 29149 tablet_replica.cc:330] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: stopping tablet replica
I20260928 14:38:19.228978 29149 master.cc:552] Master@127.22.13.125:43783 shutdown complete.
I20260928 14:38:19.229043 29149 master_runner.cc:305] Clearing existing system tablet
I20260928 14:38:19.229180 29149 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:19.229231 29149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:19.230019 29149 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 14:38:19.230592 29249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.230758 29149 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:19.230825 29149 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.230852 29149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/wal
uuid: "ad5654dbe1244b828d6054a73f683c75"
format_stamp: "Formatted at 2026-09-28 14:38:19 on dist-test-slave-wwwb"
server_key: "b22756afebdbdc7ff104386043f3f1e3"
server_key_iv: "41a7fad7e56d35bf259abac286eae60d"
server_key_version: "encryptionkey@0"
I20260928 14:38:19.231673 29149 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:19.233201 29149 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260928 14:38:19.233275 29149 log.cc:1196] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260928 14:38:19.233477 29149 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: Deleting consensus metadata
I20260928 14:38:19.233814 29149 master_runner.cc:315] Copying system tablet from 127.22.13.126:36117
I20260928 14:38:19.234484 29149 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: tablet copy: Beginning tablet copy session from remote peer at address 127.22.13.126:36117
I20260928 14:38:19.241050 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.234825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:46178)
0928 14:38:19.234968 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.234979 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.235052 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.235206 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.235207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.235361 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.235419 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.235421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.236355 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.236355 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.236881 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.236882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.237359 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.237360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.237409 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.237915 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.237919 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.238311 (+ 392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.240010 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.240011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.240012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.240197 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.240407 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.240408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.240408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.240459 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.240760 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.240761 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.240802 (+ 41us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.240865 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.240954 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":104,"threads_started":1}
I20260928 14:38:19.241097 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.234918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46178 (local address 127.22.13.126:36117)
0928 14:38:19.234972 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.234974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.235038 (+ 64us) server_negotiation.cc:407] Connection header received
0928 14:38:19.235126 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.235127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.235138 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.235195 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.235497 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.236265 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.236966 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.237281 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.237354 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.238403 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.238404 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.238405 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.238414 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.239918 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.240273 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.240274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.240275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.240341 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.240525 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.240527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.240527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.240693 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.240768 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.240916 (+ 148us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.241002 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260928 14:38:19.241475 28705 tablet_copy_service.cc:140] P 8c3817afa240422eb13936fc7717656b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ad5654dbe1244b828d6054a73f683c75 ({username='oryx', principal='oryx/127.22.13.125@KRBTEST.COM'} at 127.0.0.1:46178)
I20260928 14:38:19.241591 28705 tablet_copy_service.cc:161] P 8c3817afa240422eb13936fc7717656b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ad5654dbe1244b828d6054a73f683c75 at {username='oryx', principal='oryx/127.22.13.125@KRBTEST.COM'} at 127.0.0.1:46178: session id = ad5654dbe1244b828d6054a73f683c75-00000000000000000000000000000000
I20260928 14:38:19.242333 28705 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 14:38:19.243076 29149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260928 14:38:19.244490 29149 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: tablet copy: Starting download of 0 data blocks...
I20260928 14:38:19.244648 29149 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: tablet copy: Starting download of 1 WAL segments...
I20260928 14:38:19.245599 29149 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 14:38:19.246659 28705 tablet_copy_service.cc:342] P 8c3817afa240422eb13936fc7717656b: Request end of tablet copy session ad5654dbe1244b828d6054a73f683c75-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.13.125@KRBTEST.COM'} at 127.0.0.1:46178
I20260928 14:38:19.246752 28705 tablet_copy_service.cc:433] P 8c3817afa240422eb13936fc7717656b: ending tablet copy session ad5654dbe1244b828d6054a73f683c75-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ad5654dbe1244b828d6054a73f683c75
I20260928 14:38:19.249471 29149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:19.250644 29257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:19.250717 29259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:19.250877 29256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:19.254169 29149 server_base.cc:1034] running on GCE node
I20260928 14:38:19.254323 29149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:19.254514 29149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:19.255648 29149 hybrid_clock.cc:648] HybridClock initialized: now 1790606299255643 us; error 30 us; skew 500 ppm
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:19.257177 29149 init.cc:377] Logged in from keytab as oryx/127.22.13.125@KRBTEST.COM (short username oryx)
I20260928 14:38:19.257509 29149 webserver.cc:460] Webserver started at http://127.22.13.125:42441/ using document root <none> and password file <none>
I20260928 14:38:19.257656 29149 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:19.257695 29149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:19.258430 29149 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:19.258993 29266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.259119 29149 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:19.259166 29149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/wal
uuid: "ad5654dbe1244b828d6054a73f683c75"
format_stamp: "Formatted at 2026-09-28 14:38:19 on dist-test-slave-wwwb"
server_key: "b22756afebdbdc7ff104386043f3f1e3"
server_key_iv: "41a7fad7e56d35bf259abac286eae60d"
server_key_version: "encryptionkey@0"
I20260928 14:38:19.259230 29149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:19.272943 29149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:19.274654 29149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:19.277756 29149 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.125:43783
I20260928 14:38:19.277984 29318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.125:43783 every 8 connection(s)
I20260928 14:38:19.278189 29149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-1/data/info.pb
I20260928 14:38:19.278973 29319 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:38:19.279549 29319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: Bootstrap starting.
I20260928 14:38:19.282208 29319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:38:19.282462 29319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: Bootstrap complete.
I20260928 14:38:19.282740 29319 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: NON_VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: true } }
I20260928 14:38:19.282829 29319 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ad5654dbe1244b828d6054a73f683c75, State: Initialized, Role: LEARNER
I20260928 14:38:19.282866 29319 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: NON_VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: true } }
I20260928 14:38:19.283073 29319 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:19.284497 29323 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: NON_VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: true } } }
I20260928 14:38:19.284838 29323 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: This master's current role is: LEARNER
I20260928 14:38:19.285209 29334 catalog_manager.cc:1259] Loaded cluster ID: a0d4d515ab95487f91125cdd920f6af3
I20260928 14:38:19.285265 29334 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: loading cluster ID for follower catalog manager: success
I20260928 14:38:19.287511 29334 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: acquiring CA information for follower catalog manager: success
I20260928 14:38:19.287894 29334 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:38:19.288861 29321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.279276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44346 (local address 127.22.13.125:43783)
0928 14:38:19.280643 (+ 1367us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.280645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.280655 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:38:19.280675 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.280676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.280685 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.280793 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.281533 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.282572 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.283782 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.284171 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.284245 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.285908 (+ 1663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.285909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.285910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.285917 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.287551 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.287967 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.287969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.287969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.288032 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.288278 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.288279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.288280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.288402 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.288514 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.288648 (+ 134us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.288744 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1342,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for oryx/127.22.13.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.13.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.13.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.124.keytab.
Entry for principal oryx/127.22.13.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.124.keytab.
I20260928 14:38:19.313238 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.124:33279
--webserver_interface=127.22.13.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.124.keytab
--principal=oryx/127.22.13.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:36117,127.22.13.125:43783,127.22.13.124:33279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:19.397712 29337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:19.398003 29337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:19.398066 29337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:19.399326 29337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:19.399405 29337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:19.399438 29337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:19.399459 29337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:19.399488 29337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:19.401057 29337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:36117,127.22.13.125:43783,127.22.13.124:33279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.124.keytab
--principal=oryx/127.22.13.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.124:33279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.13.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:19.401360 29337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:19.401621 29337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:19.406445 29343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:19.406482 29345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:19.406453 29342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:19.406677 29337 server_base.cc:1034] running on GCE node
I20260928 14:38:19.407042 29337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:19.407349 29337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:19.408505 29337 hybrid_clock.cc:648] HybridClock initialized: now 1790606299408476 us; error 44 us; skew 500 ppm
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:19.411228 29337 init.cc:377] Logged in from keytab as oryx/127.22.13.124@KRBTEST.COM (short username oryx)
I20260928 14:38:19.411682 29337 webserver.cc:460] Webserver started at http://127.22.13.124:41261/ using document root <none> and password file <none>
I20260928 14:38:19.411866 29337 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:19.411906 29337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:19.411998 29337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:19.413028 29337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/data/instance:
uuid: "1a54d60db2c04cc0888795eeca75ff4a"
format_stamp: "Formatted at 2026-09-28 14:38:19 on dist-test-slave-wwwb"
server_key: "baeb6b30b1487c12ab3e69cb7687c797"
server_key_iv: "782b93d5ef65d7867115df4a7bb21dfb"
server_key_version: "encryptionkey@0"
I20260928 14:38:19.413380 29337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/wal/instance:
uuid: "1a54d60db2c04cc0888795eeca75ff4a"
format_stamp: "Formatted at 2026-09-28 14:38:19 on dist-test-slave-wwwb"
server_key: "baeb6b30b1487c12ab3e69cb7687c797"
server_key_iv: "782b93d5ef65d7867115df4a7bb21dfb"
server_key_version: "encryptionkey@0"
I20260928 14:38:19.414812 29337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:19.415535 29352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.415727 29337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:19.415815 29337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/wal
uuid: "1a54d60db2c04cc0888795eeca75ff4a"
format_stamp: "Formatted at 2026-09-28 14:38:19 on dist-test-slave-wwwb"
server_key: "baeb6b30b1487c12ab3e69cb7687c797"
server_key_iv: "782b93d5ef65d7867115df4a7bb21dfb"
server_key_version: "encryptionkey@0"
I20260928 14:38:19.415877 29337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:19.448153 29337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:19.450343 29337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:19.450472 29337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:19.456811 29337 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.124:33279
I20260928 14:38:19.456943 29404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.124:33279 every 8 connection(s)
I20260928 14:38:19.457371 29337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/data/info.pb
I20260928 14:38:19.458552 29405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:19.459116 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 29337
I20260928 14:38:19.461840 29405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.13.124@KRBTEST.COM
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.124@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM
I20260928 14:38:19.471305 29407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.459808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44148 (local address 127.22.13.124:33279)
0928 14:38:19.460106 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.460115 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.460143 (+ 28us) server_negotiation.cc:407] Connection header received
0928 14:38:19.460606 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.460613 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.460664 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.460785 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.461754 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.462773 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.463960 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.464494 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.464614 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.467559 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.467566 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.467566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.467586 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.469707 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.470142 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.470147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.470151 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.470226 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.470430 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.470431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.470431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.470667 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.470817 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.470924 (+ 107us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.471049 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":206,"threads_started":1}
I20260928 14:38:19.472585 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.463009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46182 (local address 127.22.13.126:36117)
0928 14:38:19.463086 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.463090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.463235 (+ 145us) server_negotiation.cc:407] Connection header received
0928 14:38:19.463299 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.463299 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.463308 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.463358 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.464348 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.465111 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.466559 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.466868 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.466939 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.469869 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.469870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.469871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.469879 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.471365 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.471785 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.471786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.471787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.471845 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.472045 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.472046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.472046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.472159 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.472237 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.472414 (+ 177us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.472493 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 14:38:19.472702 29408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.462721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:46182)
0928 14:38:19.463113 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.463156 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.463241 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.463450 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.463453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.464061 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.464252 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.464254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.465317 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.465318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.466476 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.466478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.467048 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.467049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.467200 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.467879 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.467883 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.469773 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.471461 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.471468 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.471471 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.471705 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.471920 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.471920 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.471921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.471983 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.472230 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.472236 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.472284 (+ 48us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.472375 (+ 91us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.472493 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":241,"threads_started":1}
I20260928 14:38:19.473140 29405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.124@KRBTEST.COM for oryx/127.22.13.125@KRBTEST.COM
I20260928 14:38:19.479296 29408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.473404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:43783 (local address 127.0.0.1:44356)
0928 14:38:19.473446 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.473456 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.473493 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.473636 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.473637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.473754 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.473789 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.473790 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.474520 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.474522 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.475029 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.475030 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.475398 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.475399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.475440 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.475872 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.475876 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.477093 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.478482 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.478483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.478483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.478651 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.478821 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.478822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.478822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.478871 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.479070 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.479071 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.479107 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.479155 (+ 48us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.479229 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260928 14:38:19.479303 29321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.473474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44356 (local address 127.22.13.125:43783)
0928 14:38:19.473514 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.473516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.473524 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:38:19.473558 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.473558 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.473585 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.473628 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.473847 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.474438 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.475128 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.475327 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.475394 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.477175 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.477183 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.477183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.477189 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.478408 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.478716 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.478717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.478717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.478766 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.478921 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.478921 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.478921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.479019 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.479068 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.479171 (+ 103us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.479241 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:38:19.479846 29405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.124@KRBTEST.COM for oryx/127.22.13.124@KRBTEST.COM
I20260928 14:38:19.487246 29408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.480104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:33279 (local address 127.0.0.1:44152)
0928 14:38:19.480154 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.480165 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.480207 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.480313 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.480314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.480443 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.480470 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.480471 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.481154 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.481156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.481688 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.481690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.482317 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.482318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.482393 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.482797 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.482802 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.484472 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.486309 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.486310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.486311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.486471 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.486692 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.486692 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.486693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.486760 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.486978 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.486979 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.487015 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.487066 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.487149 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 14:38:19.487293 29407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.480181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44152 (local address 127.22.13.124:33279)
0928 14:38:19.480216 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.480217 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.480225 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:38:19.480257 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.480258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.480266 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.480303 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.480549 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.481074 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.481794 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.482195 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.482264 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.484604 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.484605 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.484606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.484612 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.486234 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.486540 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.486541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.486541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.486589 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.486813 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.486814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.486814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.486921 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.486974 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.487082 (+ 108us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.487194 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":19}
I20260928 14:38:19.489210 29405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: Bootstrap starting.
I20260928 14:38:19.490825 29405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:19.491498 29405 log.cc:824] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:19.492429 29405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: No bootstrap required, opened a new log
I20260928 14:38:19.495152 29405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } }
I20260928 14:38:19.495316 29405 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:19.495365 29405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a54d60db2c04cc0888795eeca75ff4a, State: Initialized, Role: FOLLOWER
I20260928 14:38:19.495536 29405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } }
I20260928 14:38:19.495868 29405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:19.495918 29410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } } }
I20260928 14:38:19.496028 29410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [sys.catalog]: This master's current role is: FOLLOWER
W20260928 14:38:19.498145 29421 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:38:19.498210 29421 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:38:19.503419 29337 master_runner.cc:428] Detected that this master 1a54d60db2c04cc0888795eeca75ff4a is joining an existing cluster
I20260928 14:38:19.503484 29337 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.13.124:33279
I20260928 14:38:19.512482 29321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.504942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44366 (local address 127.22.13.125:43783)
0928 14:38:19.504991 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.504994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.505261 (+ 267us) server_negotiation.cc:407] Connection header received
0928 14:38:19.505440 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.505441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.505452 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.505506 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.506033 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.506852 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.508027 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.508339 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.508392 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.509606 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.509607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.509608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.509615 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.511243 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.511637 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.511638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.511638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.511693 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.511870 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.511872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.511872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.512022 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.512091 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.512281 (+ 190us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.512368 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:38:19.512838 29427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.504958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:43783 (local address 127.0.0.1:44366)
0928 14:38:19.505171 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.505207 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.505312 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.505579 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.505580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.505808 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.505906 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.505908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.507021 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.507022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.507930 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.507933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.508485 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.508485 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.508559 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.509076 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.509081 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.509495 (+ 414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.511337 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.511338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.511338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.511553 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.511761 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.511762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.511762 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.511815 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.512118 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.512119 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.512161 (+ 42us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.512621 (+ 460us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.512738 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":163,"threads_started":1}
I20260928 14:38:19.514253 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.505583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46198 (local address 127.22.13.126:36117)
0928 14:38:19.506362 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.506365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.506499 (+ 134us) server_negotiation.cc:407] Connection header received
0928 14:38:19.506591 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.506592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.506603 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.506633 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.507107 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.507881 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.509365 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.509697 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.509760 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.511419 (+ 1659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.511421 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.511421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.511429 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.513055 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.513434 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.513435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.513435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.513488 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.513679 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.513680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.513680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.513805 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.513846 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.514072 (+ 226us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.514161 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753}
I20260928 14:38:19.514650 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.505649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:46198)
0928 14:38:19.506434 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.506446 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.506522 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.506717 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.506718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.506921 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.507000 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.507003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.507999 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.507999 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.509279 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.509282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.509814 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.509815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.509880 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.510944 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.510949 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.511318 (+ 369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.513148 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.513148 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.513149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.513352 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.513557 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.513558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.513558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.513610 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.513889 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.513889 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.513927 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.514372 (+ 445us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.514547 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"thread_start_us":96,"threads_started":1}
I20260928 14:38:19.517525 29429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.505980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:33279 (local address 127.0.0.1:44160)
0928 14:38:19.509416 (+ 3436us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.509426 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.509486 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.509621 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.509621 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.509812 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.510246 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.510250 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.511485 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.511486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.513407 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.513409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.513772 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.513773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.513835 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.514273 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.514278 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.514738 (+ 460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.516372 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.516373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.516373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.516583 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.516888 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.516889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.516889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.516953 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.517170 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.517171 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.517226 (+ 55us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.517299 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.517403 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3420,"thread_start_us":65,"threads_started":1}
I20260928 14:38:19.517527 29407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.505881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44160 (local address 127.22.13.124:33279)
0928 14:38:19.505917 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.505919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.509466 (+ 3547us) server_negotiation.cc:407] Connection header received
0928 14:38:19.509543 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.509544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.509551 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.509576 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.510671 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.511404 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.513477 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.513685 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.513762 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.514843 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.514844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.514844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.514850 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.516288 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.516775 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.516776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.516776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.516825 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.517019 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.517020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.517020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.517111 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.517156 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.517322 (+ 166us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.517446 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":18}
I20260928 14:38:19.522660 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 28747
I20260928 14:38:19.527904 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:41419
--local_ip_for_outbound_sockets=127.22.13.65
--tserver_master_addrs=127.22.13.126:36117
--webserver_port=39501
--webserver_interface=127.22.13.65
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.65.keytab
--principal=oryx/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.13.126:36117,127.22.13.125:43783
--tserver_master_addrs=127.22.13.126:36117,127.22.13.125:43783,127.22.13.124:33279 with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606298, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.126@KRBTEST.COM for oryx/127.22.13.124@KRBTEST.COM
I20260928 14:38:19.544067 29407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.535838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44172 (local address 127.22.13.124:33279)
0928 14:38:19.535904 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.535906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.535915 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:38:19.535954 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.535955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.535966 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.536030 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.536426 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.537184 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.538399 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.538711 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.538783 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.541082 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.541084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.541085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.541093 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.542795 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.543218 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.543219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.543219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.543281 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.543493 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.543494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.543494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.543636 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.543717 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.543881 (+ 164us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.543966 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:38:19.544082 29243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.535623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:33279 (local address 127.0.0.1:44172)
0928 14:38:19.535695 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.535707 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.535753 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.536055 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.536056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.536266 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.536338 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.536340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.537279 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.537281 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.538290 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.538295 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.538792 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.538793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.538870 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.539326 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.539331 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.540964 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.542900 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.542901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.542902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.543139 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.543353 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.543354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.543354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.543427 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.543715 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.543716 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.543771 (+ 55us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.543847 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.543970 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260928 14:38:19.544600 28685 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.13.124:33279: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a"
member_type: NON_VOTER
last_known_addr {
host: "127.22.13.124"
port: 33279
}
attrs {
promote: true
}
}
dest_uuid: "8c3817afa240422eb13936fc7717656b"
cas_config_opid_index: 5
I20260928 14:38:19.544848 28685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: NON_VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: true } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } }
I20260928 14:38:19.545254 29244 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1a54d60db2c04cc0888795eeca75ff4a (127.22.13.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: NON_VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: true } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } } }
I20260928 14:38:19.545419 29379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:19.546326 29244 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.13.124:33279
I20260928 14:38:19.546399 29244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8c3817afa240422eb13936fc7717656b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: NON_VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: true } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } } }
I20260928 14:38:19.546459 29244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:19.546617 29379 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 1 FOLLOWER]: Refusing update from remote peer 8c3817afa240422eb13936fc7717656b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260928 14:38:19.546897 29245 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:19.547128 29337 master.cc:537] Master@127.22.13.124:33279 shutting down...
W20260928 14:38:19.548156 28672 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b -> Peer 1a54d60db2c04cc0888795eeca75ff4a (127.22.13.124:33279): Couldn't send request to peer 1a54d60db2c04cc0888795eeca75ff4a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 14:38:19.550617 29337 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:19.550717 29337 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:19.550778 29337 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: stopping tablet replica
I20260928 14:38:19.552376 29243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.545269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:43783 (local address 127.0.0.1:44378)
0928 14:38:19.547758 (+ 2489us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.547770 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.547816 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.548754 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.548755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.548757 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:19.548808 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.548810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.549755 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.549756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.551042 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.551046 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.552101 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.552102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.552171 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.552201 (+ 30us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.552253 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.552299 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2476,"thread_start_us":99,"threads_started":1}
I20260928 14:38:19.552889 29321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.546631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44378 (local address 127.22.13.125:43783)
0928 14:38:19.546962 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.546965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.548547 (+ 1582us) server_negotiation.cc:407] Connection header received
0928 14:38:19.548580 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.548582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.548605 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.548691 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:19.548883 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.549654 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.551273 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.551883 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.551990 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.552065 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.552775 (+ 710us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.552821 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305}
I20260928 14:38:19.553078 29293 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 LEARNER]: Refusing update from remote peer 8c3817afa240422eb13936fc7717656b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260928 14:38:19.553296 29245 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: NON_VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260928 14:38:19.553915 29293 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1a54d60db2c04cc0888795eeca75ff4a (127.22.13.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: NON_VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: true } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } } }
I20260928 14:38:19.555160 29323 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c3817afa240422eb13936fc7717656b. Latest consensus state: current_term: 1 leader_uuid: "8c3817afa240422eb13936fc7717656b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: NON_VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: true } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } } }
I20260928 14:38:19.555248 29323 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: This master's current role is: LEARNER
I20260928 14:38:19.555280 29323 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c3817afa240422eb13936fc7717656b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: NON_VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: true } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } } }
I20260928 14:38:19.555325 29323 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: This master's current role is: LEARNER
I20260928 14:38:19.555555 29245 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b: attempting to promote NON_VOTER ad5654dbe1244b828d6054a73f683c75 to VOTER
I20260928 14:38:19.555758 29245 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } }
I20260928 14:38:19.556331 29243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.556144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:33279 (local address 127.0.0.1:44184)
0928 14:38:19.556267 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.556305 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260928 14:38:19.556703 29243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.556624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:33279 (local address 127.0.0.1:44196)
0928 14:38:19.556654 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.556683 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:33279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260928 14:38:19.556833 28672 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b -> Peer 1a54d60db2c04cc0888795eeca75ff4a (127.22.13.124:33279): Couldn't send request to peer 1a54d60db2c04cc0888795eeca75ff4a. Status: Network error: Client connection negotiation failed: client connection to 127.22.13.124:33279: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260928 14:38:19.557070 29293 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 LEARNER]: Refusing update from remote peer 8c3817afa240422eb13936fc7717656b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260928 14:38:19.557245 29244 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260928 14:38:19.558012 29293 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ad5654dbe1244b828d6054a73f683c75 (127.22.13.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } } }
I20260928 14:38:19.559010 29433 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c3817afa240422eb13936fc7717656b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } } }
I20260928 14:38:19.559101 29433 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:38:19.557869 29434 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ad5654dbe1244b828d6054a73f683c75 (127.22.13.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } } }
I20260928 14:38:19.560176 29244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8c3817afa240422eb13936fc7717656b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } } }
I20260928 14:38:19.560259 29244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:19.560346 29434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c3817afa240422eb13936fc7717656b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } } }
I20260928 14:38:19.560396 29434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:19.566033 29337 master.cc:552] Master@127.22.13.124:33279 shutdown complete.
I20260928 14:38:19.566093 29337 master_runner.cc:305] Clearing existing system tablet
I20260928 14:38:19.566222 29337 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:19.566267 29337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:19.567036 29337 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:19.567597 29437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.567754 29337 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:19.567817 29337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:19.567842 29337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/wal
uuid: "1a54d60db2c04cc0888795eeca75ff4a"
format_stamp: "Formatted at 2026-09-28 14:38:19 on dist-test-slave-wwwb"
server_key: "baeb6b30b1487c12ab3e69cb7687c797"
server_key_iv: "782b93d5ef65d7867115df4a7bb21dfb"
server_key_version: "encryptionkey@0"
I20260928 14:38:19.568588 29337 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:19.570091 29337 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260928 14:38:19.570159 29337 log.cc:1196] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260928 14:38:19.570354 29337 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: Deleting consensus metadata
I20260928 14:38:19.570627 29337 master_runner.cc:315] Copying system tablet from 127.22.13.126:36117
I20260928 14:38:19.571239 29337 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: tablet copy: Beginning tablet copy session from remote peer at address 127.22.13.126:36117
I20260928 14:38:19.581151 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.571621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46204 (local address 127.22.13.126:36117)
0928 14:38:19.571688 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.571690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.574248 (+ 2558us) server_negotiation.cc:407] Connection header received
0928 14:38:19.574312 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.574313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.574328 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.574404 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.574792 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.575548 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.576552 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.576896 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.576984 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.578152 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.578153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.578154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.578162 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.579858 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.580289 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.580290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.580290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.580353 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.580584 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.580585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.580586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.580728 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.580792 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.580976 (+ 184us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.581059 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260928 14:38:19.581163 29443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.571540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:46204)
0928 14:38:19.574133 (+ 2593us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.574148 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.574248 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.574425 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.574426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.574651 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.574708 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.574710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.575651 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.575652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.576438 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.576441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.576992 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.576993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.577063 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.577610 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.577615 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.578058 (+ 443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.579957 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.579958 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.579959 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.580207 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.580428 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.580429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.580429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.580504 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.580806 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.580807 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.580863 (+ 56us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.580942 (+ 79us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.581063 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2564,"thread_start_us":62,"threads_started":1}
I20260928 14:38:19.581974 28705 tablet_copy_service.cc:140] P 8c3817afa240422eb13936fc7717656b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1a54d60db2c04cc0888795eeca75ff4a ({username='oryx', principal='oryx/127.22.13.124@KRBTEST.COM'} at 127.0.0.1:46204)
I20260928 14:38:19.582041 28705 tablet_copy_service.cc:161] P 8c3817afa240422eb13936fc7717656b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1a54d60db2c04cc0888795eeca75ff4a at {username='oryx', principal='oryx/127.22.13.124@KRBTEST.COM'} at 127.0.0.1:46204: session id = 1a54d60db2c04cc0888795eeca75ff4a-00000000000000000000000000000000
I20260928 14:38:19.582626 28705 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 14:38:19.583134 29337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260928 14:38:19.584513 29337 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: tablet copy: Starting download of 0 data blocks...
I20260928 14:38:19.584671 29337 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: tablet copy: Starting download of 1 WAL segments...
I20260928 14:38:19.585398 29337 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 14:38:19.586359 28705 tablet_copy_service.cc:342] P 8c3817afa240422eb13936fc7717656b: Request end of tablet copy session 1a54d60db2c04cc0888795eeca75ff4a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.13.124@KRBTEST.COM'} at 127.0.0.1:46204
I20260928 14:38:19.586416 28705 tablet_copy_service.cc:433] P 8c3817afa240422eb13936fc7717656b: ending tablet copy session 1a54d60db2c04cc0888795eeca75ff4a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1a54d60db2c04cc0888795eeca75ff4a
I20260928 14:38:19.589074 29337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:19.590207 29444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:19.590747 29445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:19.591079 29447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:19.594956 29337 server_base.cc:1034] running on GCE node
I20260928 14:38:19.595067 29337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:19.595211 29337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:19.596338 29337 hybrid_clock.cc:648] HybridClock initialized: now 1790606299596329 us; error 29 us; skew 500 ppm
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:19.597996 29337 init.cc:377] Logged in from keytab as oryx/127.22.13.124@KRBTEST.COM (short username oryx)
I20260928 14:38:19.598289 29337 webserver.cc:460] Webserver started at http://127.22.13.124:40035/ using document root <none> and password file <none>
I20260928 14:38:19.598413 29337 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:19.598452 29337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:19.599229 29337 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:19.599866 29454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.600039 29337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:19.600098 29337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/wal
uuid: "1a54d60db2c04cc0888795eeca75ff4a"
format_stamp: "Formatted at 2026-09-28 14:38:19 on dist-test-slave-wwwb"
server_key: "baeb6b30b1487c12ab3e69cb7687c797"
server_key_iv: "782b93d5ef65d7867115df4a7bb21dfb"
server_key_version: "encryptionkey@0"
I20260928 14:38:19.600178 29337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:19.623086 29337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:19.624967 29337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:19.628341 29337 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.124:33279
I20260928 14:38:19.628461 29506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.124:33279 every 8 connection(s)
I20260928 14:38:19.628712 29337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/master-2/data/info.pb
I20260928 14:38:19.629874 29507 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:38:19.630470 29507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: Bootstrap starting.
W20260928 14:38:19.632869 29430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:19.633093 29430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:19.633140 29430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260928 14:38:19.633985 29507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:38:19.634248 29507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: Bootstrap complete.
I20260928 14:38:19.634500 29507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } }
I20260928 14:38:19.634613 29507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1a54d60db2c04cc0888795eeca75ff4a, State: Initialized, Role: LEARNER
I20260928 14:38:19.634660 29507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } }
W20260928 14:38:19.634810 29430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
I20260928 14:38:19.634845 29507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [sys.catalog]: configured and running, proceeding with master startup.
W20260928 14:38:19.634868 29430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:19.634927 29430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:38:19.634928 29513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } } }
I20260928 14:38:19.635124 29513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [sys.catalog]: This master's current role is: LEARNER
I20260928 14:38:19.636341 29524 catalog_manager.cc:1259] Loaded cluster ID: a0d4d515ab95487f91125cdd920f6af3
I20260928 14:38:19.636437 29524 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: loading cluster ID for follower catalog manager: success
I20260928 14:38:19.637233 29430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.65.keytab
--principal=oryx/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:41419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=39501
--tserver_master_addrs=127.22.13.126:36117,127.22.13.125:43783,127.22.13.124:33279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:19.637503 29430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:19.637753 29430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:38:19.637790 29524 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: acquiring CA information for follower catalog manager: success
I20260928 14:38:19.638126 29524 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260928 14:38:19.643952 29527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:19.643998 29529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:19.643957 29526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:19.644435 29430 server_base.cc:1034] running on GCE node
I20260928 14:38:19.644656 29430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:19.644940 29430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:19.646111 29430 hybrid_clock.cc:648] HybridClock initialized: now 1790606299646098 us; error 37 us; skew 500 ppm
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:19.649214 29430 init.cc:377] Logged in from keytab as oryx/127.22.13.65@KRBTEST.COM (short username oryx)
I20260928 14:38:19.649693 29430 webserver.cc:460] Webserver started at http://127.22.13.65:39501/ using document root <none> and password file <none>
I20260928 14:38:19.649978 29430 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:19.650064 29430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:19.651559 29430 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:19.652280 29536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.652469 29430 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:19.652580 29430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "456ec803412d4816b65345ebf7a8d8a5"
format_stamp: "Formatted at 2026-09-28 14:38:18 on dist-test-slave-wwwb"
server_key: "75c78655e475dc387a77e5d849c9610b"
server_key_iv: "83a1554d502c32218cde2d2b7bd1bdaf"
server_key_version: "encryptionkey@0"
I20260928 14:38:19.653035 29430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:19.670706 29430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:19.673195 29430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:19.673372 29430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:19.673631 29430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:19.674010 29430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:19.674065 29430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.674108 29430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:19.674134 29430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.680439 29430 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:41419
I20260928 14:38:19.680526 29649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:41419 every 8 connection(s)
I20260928 14:38:19.680903 29430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:38:19.686328 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 29430
I20260928 14:38:19.686489 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 28881
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.125@KRBTEST.COM
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM
I20260928 14:38:19.693377 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.682030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:54617 (local address 127.22.13.126:36117)
0928 14:38:19.682071 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.682073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.682127 (+ 54us) server_negotiation.cc:407] Connection header received
0928 14:38:19.682722 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.682723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.682737 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.682770 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.683739 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.684551 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.686030 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.686282 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.686361 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.689887 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.689888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.689888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.689896 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.691401 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.691942 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.691944 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.691944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.692004 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.692574 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.692575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.692575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.692711 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.692758 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.693186 (+ 428us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.693272 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:38:19.693377 29321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.681736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:59807 (local address 127.22.13.125:43783)
0928 14:38:19.681885 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.681887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.682314 (+ 427us) server_negotiation.cc:407] Connection header received
0928 14:38:19.682722 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.682723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.682737 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.682770 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.683743 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.684530 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.685969 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.686266 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.686339 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.689664 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.689666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.689666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.689675 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.691756 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.692174 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.692175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.692176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.692235 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.692560 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.692561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.692561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.692717 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.692758 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.693181 (+ 423us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.693276 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.124@KRBTEST.COM
I20260928 14:38:19.694046 29657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.681707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.65:54617)
0928 14:38:19.682065 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.682076 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.682631 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.683365 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.683366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.683540 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.683607 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.683614 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.684679 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.684680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.685914 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.685916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.686414 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.686414 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.686502 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.687308 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.687320 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.689800 (+ 2480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.691543 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.691553 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.691555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.691837 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.692082 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.692083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.692083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.692471 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.692811 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.692820 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.693013 (+ 193us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.693541 (+ 528us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.693769 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":161,"thread_start_us":86,"threads_started":1}
I20260928 14:38:19.694051 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:36965
--local_ip_for_outbound_sockets=127.22.13.66
--tserver_master_addrs=127.22.13.126:36117
--webserver_port=39631
--webserver_interface=127.22.13.66
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.66.keytab
--principal=oryx/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.13.126:36117,127.22.13.125:43783
--tserver_master_addrs=127.22.13.126:36117,127.22.13.125:43783,127.22.13.124:33279 with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:38:19.694617 29654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.681708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:43783 (local address 127.22.13.65:59807)
0928 14:38:19.682244 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.682252 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.682618 (+ 366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.682880 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.682881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.683351 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.683608 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.683614 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.684678 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.684679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.685823 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.685827 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.686357 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.686358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.686502 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.687310 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.687320 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.689564 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.691845 (+ 2281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.691845 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.691846 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.692080 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.692311 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.692312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.692312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.692471 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.692811 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.692820 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.693023 (+ 203us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.693923 (+ 900us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.694070 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":421,"thread_start_us":68,"threads_started":1}
I20260928 14:38:19.694823 29651 heartbeater.cc:347] Connected to a master server at 127.22.13.126:36117
I20260928 14:38:19.695147 29651 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:19.695416 29651 heartbeater.cc:511] Master 127.22.13.126:36117 requested a full tablet report, sending...
I20260928 14:38:19.695873 28685 ts_manager.cc:194] Re-registered known tserver with Master: 456ec803412d4816b65345ebf7a8d8a5 (127.22.13.65:41419)
I20260928 14:38:19.695956 29652 heartbeater.cc:347] Connected to a master server at 127.22.13.125:43783
I20260928 14:38:19.696024 29652 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:19.696099 29652 heartbeater.cc:511] Master 127.22.13.125:43783 requested a full tablet report, sending...
I20260928 14:38:19.696401 28685 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:54617
I20260928 14:38:19.696533 29283 ts_manager.cc:194] Registered new tserver with Master: 456ec803412d4816b65345ebf7a8d8a5 (127.22.13.65:41419)
I20260928 14:38:19.697219 29655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.681707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:33279 (local address 127.22.13.65:34199)
0928 14:38:19.681912 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.681983 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.682595 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.682866 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.682871 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.683338 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.683606 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.683611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.684729 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.684730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.685770 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.685772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.691868 (+ 6096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.691869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.691971 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.692395 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.692401 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.693972 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.695954 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.695955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.695955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.696252 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.696468 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.696469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.696469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.696528 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.696827 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.696828 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.696873 (+ 45us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.696938 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.697122 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":111,"thread_start_us":95,"threads_started":1}
I20260928 14:38:19.697264 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.681787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:34199 (local address 127.22.13.124:33279)
0928 14:38:19.681962 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.681964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.682204 (+ 240us) server_negotiation.cc:407] Connection header received
0928 14:38:19.682698 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.682699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.682714 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.682791 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.683747 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.684640 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.685863 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.690932 (+ 5069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.692128 (+ 1196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.694079 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.694080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.694081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.694089 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.695853 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.696335 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.696336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.696337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.696400 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.696597 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.696598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.696598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.696740 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.696802 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.697011 (+ 209us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.697108 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":53,"threads_started":1}
I20260928 14:38:19.697861 29650 heartbeater.cc:347] Connected to a master server at 127.22.13.124:33279
I20260928 14:38:19.697921 29650 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:19.698076 29650 heartbeater.cc:511] Master 127.22.13.124:33279 requested a full tablet report, sending...
I20260928 14:38:19.698477 29471 ts_manager.cc:194] Registered new tserver with Master: 456ec803412d4816b65345ebf7a8d8a5 (127.22.13.65:41419)
W20260928 14:38:19.778177 29658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:19.778443 29658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:19.778486 29658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:19.779870 29658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:19.779932 29658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:19.779979 29658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:38:19.781606 29658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.66.keytab
--principal=oryx/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:36965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=39631
--tserver_master_addrs=127.22.13.126:36117,127.22.13.125:43783,127.22.13.124:33279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:19.781905 29658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:19.782249 29658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:19.787956 29664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:19.787963 29666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:19.787968 29663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:19.788153 29658 server_base.cc:1034] running on GCE node
I20260928 14:38:19.788349 29658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:19.788565 29658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:19.789760 29658 hybrid_clock.cc:648] HybridClock initialized: now 1790606299789757 us; error 36 us; skew 500 ppm
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:19.792307 29658 init.cc:377] Logged in from keytab as oryx/127.22.13.66@KRBTEST.COM (short username oryx)
I20260928 14:38:19.792809 29658 webserver.cc:460] Webserver started at http://127.22.13.66:39631/ using document root <none> and password file <none>
I20260928 14:38:19.793058 29658 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:19.793147 29658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:19.794668 29658 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:38:19.795274 29673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.795445 29658 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:19.795552 29658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "d844e25249cd4df5a06f0884bf9850f5"
format_stamp: "Formatted at 2026-09-28 14:38:18 on dist-test-slave-wwwb"
server_key: "51c5baba3ba5b84408efeafe5f9ff815"
server_key_iv: "902f081161520b6adc58a2cfd154e774"
server_key_version: "encryptionkey@0"
I20260928 14:38:19.795917 29658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:19.814248 29658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:19.816183 29658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:19.816358 29658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:19.816612 29658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:19.816934 29658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:19.816990 29658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.817035 29658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:19.817065 29658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.823189 29658 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:36965
I20260928 14:38:19.823235 29786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:36965 every 8 connection(s)
I20260928 14:38:19.823637 29658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:38:19.830519 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 29658
I20260928 14:38:19.830696 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 29015
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.124@KRBTEST.COM
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.125@KRBTEST.COM
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM
I20260928 14:38:19.836042 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.824534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:38719 (local address 127.22.13.124:33279)
0928 14:38:19.824624 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.824626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.825051 (+ 425us) server_negotiation.cc:407] Connection header received
0928 14:38:19.825860 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.825861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.825880 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.825913 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.826781 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.827562 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.828473 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.828755 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.828818 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.832752 (+ 3934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.832753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.832753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.832762 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.834606 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.835025 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.835026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.835026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.835083 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.835332 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.835333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.835333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.835459 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.835498 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.835821 (+ 323us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.835923 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260928 14:38:19.836064 29321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.824479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:45327 (local address 127.22.13.125:43783)
0928 14:38:19.824589 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.824591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.824852 (+ 261us) server_negotiation.cc:407] Connection header received
0928 14:38:19.825918 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.825919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.825952 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.826014 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.826780 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.827556 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.828996 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.829169 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.829636 (+ 467us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.832141 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.832143 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.832143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.832151 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.834131 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.834746 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.834747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.834748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.834831 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.835125 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.835126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.835126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.835343 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.835402 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.835903 (+ 501us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.835979 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260928 14:38:19.836647 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.824468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:43783 (local address 127.22.13.66:45327)
0928 14:38:19.824761 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.824771 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.825763 (+ 992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.826068 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.826069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.826429 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.826640 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.826646 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.827638 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.827639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.828536 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.828538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.829636 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.829637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.829705 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.830119 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.830124 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.832033 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.834234 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.834243 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.834246 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.834600 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.834903 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.834904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.834905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.835053 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.835427 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.835434 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.835630 (+ 196us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.835808 (+ 178us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.836384 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":37,"thread_start_us":88,"threads_started":1}
I20260928 14:38:19.836652 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:38887
--local_ip_for_outbound_sockets=127.22.13.67
--tserver_master_addrs=127.22.13.126:36117
--webserver_port=43743
--webserver_interface=127.22.13.67
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.67.keytab
--principal=oryx/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.13.126:36117,127.22.13.125:43783
--tserver_master_addrs=127.22.13.126:36117,127.22.13.125:43783,127.22.13.124:33279 with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:38:19.836647 29793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.824467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:33279 (local address 127.22.13.66:38719)
0928 14:38:19.824970 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.824978 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.825764 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.826049 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.826050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.826436 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.826640 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.826646 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.827655 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.827656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.828381 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.828382 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.828833 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.828834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.828960 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.829621 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.829631 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.832651 (+ 3020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.834714 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.834715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.834715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.834935 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.835185 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.835186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.835187 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.835236 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.835542 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.835543 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.835630 (+ 87us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.836125 (+ 495us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.836431 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":361,"thread_start_us":66,"threads_started":1}
I20260928 14:38:19.838541 29789 heartbeater.cc:347] Connected to a master server at 127.22.13.125:43783
I20260928 14:38:19.838649 29789 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:19.838725 29787 heartbeater.cc:347] Connected to a master server at 127.22.13.124:33279
I20260928 14:38:19.838778 29787 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:19.838811 29789 heartbeater.cc:511] Master 127.22.13.125:43783 requested a full tablet report, sending...
I20260928 14:38:19.838876 29787 heartbeater.cc:511] Master 127.22.13.124:33279 requested a full tablet report, sending...
I20260928 14:38:19.839274 29471 ts_manager.cc:194] Registered new tserver with Master: d844e25249cd4df5a06f0884bf9850f5 (127.22.13.66:36965)
I20260928 14:38:19.839474 29283 ts_manager.cc:194] Registered new tserver with Master: d844e25249cd4df5a06f0884bf9850f5 (127.22.13.66:36965)
I20260928 14:38:19.840171 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.824597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:42769 (local address 127.22.13.126:36117)
0928 14:38:19.824643 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.824645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.824852 (+ 207us) server_negotiation.cc:407] Connection header received
0928 14:38:19.825862 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.825864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.825878 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.825928 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.826811 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.827313 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.828619 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.828899 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.828947 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.835937 (+ 6990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.835939 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.835939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.835947 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.838622 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.839225 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.839227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.839227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.839292 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.839582 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.839583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.839583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.839728 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.839772 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.839988 (+ 216us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.840063 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:38:19.840195 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.824467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.66:42769)
0928 14:38:19.824690 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.824740 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.825738 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.826039 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.826043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.826413 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.826638 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.826642 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.827432 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.827433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.828533 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.828535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.828963 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.828964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.829016 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.829621 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.829631 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.835831 (+ 6200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.838880 (+ 3049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.838881 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.838881 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.839128 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.839408 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.839408 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.839409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.839468 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.839817 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.839818 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.839859 (+ 41us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.839949 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.840064 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":5,"thread_start_us":99,"threads_started":1}
I20260928 14:38:19.840726 29788 heartbeater.cc:347] Connected to a master server at 127.22.13.126:36117
I20260928 14:38:19.840775 29788 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:19.840844 29788 heartbeater.cc:511] Master 127.22.13.126:36117 requested a full tablet report, sending...
I20260928 14:38:19.841048 28685 ts_manager.cc:194] Re-registered known tserver with Master: d844e25249cd4df5a06f0884bf9850f5 (127.22.13.66:36965)
I20260928 14:38:19.841454 28685 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:42769
W20260928 14:38:19.919845 29794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:19.920133 29794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:19.920157 29794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:19.921355 29794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:19.921399 29794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:19.921427 29794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:38:19.922981 29794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.67.keytab
--principal=oryx/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:38887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=43743
--tserver_master_addrs=127.22.13.126:36117,127.22.13.125:43783,127.22.13.124:33279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:19.923259 29794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:19.923506 29794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:19.928725 29799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:19.928717 29800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:19.928848 29794 server_base.cc:1034] running on GCE node
W20260928 14:38:19.928695 29802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:19.929167 29794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:19.929462 29794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:19.930632 29794 hybrid_clock.cc:648] HybridClock initialized: now 1790606299930620 us; error 43 us; skew 500 ppm
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:19.933254 29794 init.cc:377] Logged in from keytab as oryx/127.22.13.67@KRBTEST.COM (short username oryx)
I20260928 14:38:19.933698 29794 webserver.cc:460] Webserver started at http://127.22.13.67:43743/ using document root <none> and password file <none>
I20260928 14:38:19.933928 29794 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:19.934067 29794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:19.935603 29794 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:19.936236 29809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.936403 29794 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:38:19.936544 29794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "ac34180e0f1c455da81fb0c4bc578d4b"
format_stamp: "Formatted at 2026-09-28 14:38:18 on dist-test-slave-wwwb"
server_key: "82add26c79ef89108a7fe2920644c2c2"
server_key_iv: "602316d083025aaaad7618e74d3d19af"
server_key_version: "encryptionkey@0"
I20260928 14:38:19.936898 29794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:19.948302 29794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:19.950138 29794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:19.950304 29794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:19.950532 29794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:19.950841 29794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:19.950896 29794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.950937 29794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:19.950964 29794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:19.956959 29794 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:38887
I20260928 14:38:19.957006 29922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:38887 every 8 connection(s)
I20260928 14:38:19.957489 29794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:38:19.962838 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 29794
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.125@KRBTEST.COM
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM
Sep 28 14:38:19 dist-test-slave-wwwb krb5kdc[28640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606299, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.124@KRBTEST.COM
I20260928 14:38:19.970963 29321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.958397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:47249 (local address 127.22.13.125:43783)
0928 14:38:19.958444 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.958446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.958896 (+ 450us) server_negotiation.cc:407] Connection header received
0928 14:38:19.959383 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.959385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.959396 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.959426 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.960479 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.961091 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.962255 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.962543 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.962605 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.966839 (+ 4234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.966840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.966840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.966847 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.968623 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.969862 (+ 1239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.969863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.969863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.969919 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.970160 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.970160 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.970161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.970290 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.970356 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.970758 (+ 402us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.970843 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:38:19.971140 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.958265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.67:45729)
0928 14:38:19.958539 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.958588 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.959231 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.959537 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.959541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.960076 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.960384 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.960390 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.961370 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.961371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.962311 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.962313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.962797 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.962798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.962853 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.963648 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.963659 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.966378 (+ 2719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.969406 (+ 3028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.969407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.969407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.969651 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.969865 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.969865 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.969866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.970038 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.970334 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.970341 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.970557 (+ 216us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.970658 (+ 101us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.970857 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":37,"thread_start_us":87,"threads_started":1}
I20260928 14:38:19.971594 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.958265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:43783 (local address 127.22.13.67:47249)
0928 14:38:19.958816 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.958825 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.959246 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.959916 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.959917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.960100 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.960384 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.960389 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.961227 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.961228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.962165 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.962168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.962675 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.962676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.962830 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.963668 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.963672 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.966748 (+ 3076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.968714 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.968722 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.968725 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.969052 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.970025 (+ 973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.970025 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.970026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.970088 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.970385 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.970386 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.970550 (+ 164us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.970673 (+ 123us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.971467 (+ 794us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":842,"thread_start_us":61,"threads_started":1}
I20260928 14:38:19.971971 29925 heartbeater.cc:347] Connected to a master server at 127.22.13.125:43783
I20260928 14:38:19.972075 29925 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:19.972254 29928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.958265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:33279 (local address 127.22.13.67:48153)
0928 14:38:19.958559 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.958588 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.959231 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.959537 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.959541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.960080 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:19.960384 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.960390 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.961370 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.961371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.962313 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.962314 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.962815 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.962816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.962884 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.963818 (+ 934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:19.963823 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:19.966748 (+ 2925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:19.971136 (+ 4388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.971137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.971138 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.971349 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.971568 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:19.971569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:19.971569 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:19.971635 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:19.971904 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:19.971905 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:19.971943 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.972017 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.972126 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":78,"thread_start_us":89,"threads_started":1}
I20260928 14:38:19.972289 29925 heartbeater.cc:511] Master 127.22.13.125:43783 requested a full tablet report, sending...
I20260928 14:38:19.972568 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.958330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:48153 (local address 127.22.13.124:33279)
0928 14:38:19.958390 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.958392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.958678 (+ 286us) server_negotiation.cc:407] Connection header received
0928 14:38:19.959345 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.959346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.959360 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.959419 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.960496 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.961273 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.962417 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.962686 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.962755 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.966848 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.966849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.966850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.966855 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.969788 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.971436 (+ 1648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.971437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.971438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.971501 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.971707 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.971708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.971708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.971830 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.971881 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.972397 (+ 516us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.972485 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:38:19.972743 29283 ts_manager.cc:194] Registered new tserver with Master: ac34180e0f1c455da81fb0c4bc578d4b (127.22.13.67:38887)
I20260928 14:38:19.973065 29923 heartbeater.cc:347] Connected to a master server at 127.22.13.124:33279
I20260928 14:38:19.973119 29923 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:19.973189 29923 heartbeater.cc:511] Master 127.22.13.124:33279 requested a full tablet report, sending...
I20260928 14:38:19.973296 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.958527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:45729 (local address 127.22.13.126:36117)
0928 14:38:19.958568 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.958570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.958678 (+ 108us) server_negotiation.cc:407] Connection header received
0928 14:38:19.959411 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.959412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.959424 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.959469 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.960507 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.961272 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.962393 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.962689 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.962747 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.966471 (+ 3724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.966472 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.966473 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.966480 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.969312 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.969738 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.969739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.969739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.969796 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.970110 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.970110 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.970111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.970246 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.970309 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.973114 (+ 2805us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.973195 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:38:19.973376 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.964048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46208 (local address 127.22.13.126:36117)
0928 14:38:19.965026 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.965028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.965037 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:38:19.965061 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.965062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.965074 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.965145 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.966146 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.966882 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.968067 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.968416 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.968798 (+ 382us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.970018 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.970020 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.970020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.970026 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.971905 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.972306 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.972307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.972308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.972364 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.972573 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.972573 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.972574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.972688 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.973163 (+ 475us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.973197 (+ 34us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.973259 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":945,"thread_start_us":68,"threads_started":1}
I20260928 14:38:19.973472 29471 ts_manager.cc:194] Registered new tserver with Master: ac34180e0f1c455da81fb0c4bc578d4b (127.22.13.67:38887)
I20260928 14:38:19.973591 29924 heartbeater.cc:347] Connected to a master server at 127.22.13.126:36117
I20260928 14:38:19.973635 29924 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:19.973690 29924 heartbeater.cc:511] Master 127.22.13.126:36117 requested a full tablet report, sending...
I20260928 14:38:19.973884 28684 ts_manager.cc:194] Re-registered known tserver with Master: ac34180e0f1c455da81fb0c4bc578d4b (127.22.13.67:38887)
I20260928 14:38:19.974364 28684 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:45729
I20260928 14:38:19.977092 29937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.964256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44198 (local address 127.22.13.124:33279)
0928 14:38:19.964912 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.964914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.964923 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:38:19.964945 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.964945 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.964957 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.965023 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.967888 (+ 2865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.968701 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.969758 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.970084 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.970142 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.973319 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.973321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.973321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.973327 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.975873 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.976283 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.976285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.976285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.976342 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.976563 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.976563 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.976564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.976683 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.976787 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.976892 (+ 105us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.976972 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":53,"threads_started":1}
I20260928 14:38:19.975745 28684 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46208:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:38:19.978390 28684 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:38:19.979615 29939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.964930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44394 (local address 127.22.13.125:43783)
0928 14:38:19.965025 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.965027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.965216 (+ 189us) server_negotiation.cc:407] Connection header received
0928 14:38:19.965240 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.965241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.965254 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.965309 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:19.966137 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.973987 (+ 7850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.974880 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.975249 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.975331 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.976782 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:19.976783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:19.976783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:19.976790 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:19.978362 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.978725 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.978726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.978726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.978802 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:19.978949 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:19.978950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:19.978950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:19.979075 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:19.979279 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.979424 (+ 145us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.979505 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":50,"threads_started":1}
I20260928 14:38:19.986415 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.982237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:41419 (local address 127.0.0.1:52544)
0928 14:38:19.982338 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.982350 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.982397 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.982922 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.982922 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.982923 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:19.982990 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.982992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.984594 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.984595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.985380 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.985383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.986172 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.986173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.986233 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.986238 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.986292 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.986334 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":14}
I20260928 14:38:19.986423 29944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.982324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52544 (local address 127.22.13.65:41419)
0928 14:38:19.982761 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.982766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.982776 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:38:19.982819 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.982821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.982838 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.983263 (+ 425us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:19.983323 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.984531 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.985469 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.986077 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.986170 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.986256 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.986310 (+ 54us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.986347 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":88,"threads_started":1}
I20260928 14:38:19.986550 29243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.982276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:38887 (local address 127.0.0.1:52098)
0928 14:38:19.982448 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.982458 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.982495 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.983204 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.983204 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.983206 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:19.983235 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.983235 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.984448 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.984449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.985361 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.985364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.986338 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.986339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.986396 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.986400 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.986428 (+ 28us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.986471 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":70}
I20260928 14:38:19.986958 29946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.982355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52098 (local address 127.22.13.67:38887)
0928 14:38:19.983016 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.983020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.983028 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:38:19.983106 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.983107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.983121 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.983199 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:19.983292 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.984368 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.985466 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.986246 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.986718 (+ 472us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.986803 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.986827 (+ 24us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.986888 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":84,"threads_started":1}
I20260928 14:38:19.987500 29857 tablet_service.cc:1467] Processing CreateTablet for tablet e5c2a73f852547ac92fff707e9f09488 (DEFAULT_TABLE table=test-table [id=742b210e6326476e9d926660427f987b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:19.987491 29584 tablet_service.cc:1467] Processing CreateTablet for tablet e5c2a73f852547ac92fff707e9f09488 (DEFAULT_TABLE table=test-table [id=742b210e6326476e9d926660427f987b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:19.987779 29857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5c2a73f852547ac92fff707e9f09488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:19.987843 29584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5c2a73f852547ac92fff707e9f09488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:19.990810 29945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.982326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:36965 (local address 127.0.0.1:59998)
0928 14:38:19.982564 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:19.982574 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:19.982616 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:19.982919 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:19.982920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:19.982922 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:19.983116 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.983118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.984423 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.984424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.986438 (+ 2014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:19.986440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.990520 (+ 4080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.990522 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:19.990598 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.990604 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 14:38:19.990675 (+ 71us) client_negotiation.cc:240] Negotiation successful
0928 14:38:19.990729 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":91,"threads_started":1}
I20260928 14:38:19.990979 29947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:19.982577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59998 (local address 127.22.13.66:36965)
0928 14:38:19.982761 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:19.982766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:19.982776 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:38:19.982820 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:19.982822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:19.982838 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:19.982922 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:19.983170 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.984329 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.986540 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:19.990416 (+ 3876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:19.990503 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:19.990582 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:19.990810 (+ 228us) server_negotiation.cc:299] Negotiation successful
0928 14:38:19.990877 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":94,"threads_started":1}
I20260928 14:38:19.991056 29948 tablet_bootstrap.cc:492] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b: Bootstrap starting.
I20260928 14:38:19.991758 29948 tablet_bootstrap.cc:654] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:19.991709 29721 tablet_service.cc:1467] Processing CreateTablet for tablet e5c2a73f852547ac92fff707e9f09488 (DEFAULT_TABLE table=test-table [id=742b210e6326476e9d926660427f987b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:19.992070 29721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5c2a73f852547ac92fff707e9f09488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:19.992077 29948 log.cc:824] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:19.992409 29949 tablet_bootstrap.cc:492] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5: Bootstrap starting.
I20260928 14:38:19.992810 29948 tablet_bootstrap.cc:492] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b: No bootstrap required, opened a new log
I20260928 14:38:19.992916 29948 ts_tablet_manager.cc:1397] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:38:19.993098 29949 tablet_bootstrap.cc:654] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:19.993438 29949 log.cc:824] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:19.995057 29948 raft_consensus.cc:348] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } }
I20260928 14:38:19.995258 29948 raft_consensus.cc:374] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:19.995301 29948 raft_consensus.cc:729] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac34180e0f1c455da81fb0c4bc578d4b, State: Initialized, Role: FOLLOWER
I20260928 14:38:19.995342 29952 tablet_bootstrap.cc:492] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5: Bootstrap starting.
I20260928 14:38:19.995417 29948 consensus_queue.cc:260] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } }
I20260928 14:38:19.995659 29948 ts_tablet_manager.cc:1428] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:38:19.995980 29924 heartbeater.cc:503] Master 127.22.13.126:36117 was elected leader, sending a full tablet report...
I20260928 14:38:19.996003 29952 tablet_bootstrap.cc:654] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:19.996315 29952 log.cc:824] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:19.996989 29949 tablet_bootstrap.cc:492] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5: No bootstrap required, opened a new log
I20260928 14:38:19.997063 29949 ts_tablet_manager.cc:1397] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260928 14:38:19.997141 29952 tablet_bootstrap.cc:492] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5: No bootstrap required, opened a new log
I20260928 14:38:19.997208 29952 ts_tablet_manager.cc:1397] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:38:19.999281 29949 raft_consensus.cc:348] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } }
I20260928 14:38:19.999426 29949 raft_consensus.cc:374] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:19.999449 29949 raft_consensus.cc:729] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456ec803412d4816b65345ebf7a8d8a5, State: Initialized, Role: FOLLOWER
I20260928 14:38:19.999364 29952 raft_consensus.cc:348] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } }
I20260928 14:38:19.999573 29952 raft_consensus.cc:374] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:19.999647 29952 raft_consensus.cc:729] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d844e25249cd4df5a06f0884bf9850f5, State: Initialized, Role: FOLLOWER
I20260928 14:38:19.999610 29949 consensus_queue.cc:260] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } }
I20260928 14:38:19.999773 29952 consensus_queue.cc:260] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } }
I20260928 14:38:19.999928 29949 ts_tablet_manager.cc:1428] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:38:20.000031 29651 heartbeater.cc:503] Master 127.22.13.126:36117 was elected leader, sending a full tablet report...
I20260928 14:38:20.000046 29952 ts_tablet_manager.cc:1428] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:38:20.000329 29788 heartbeater.cc:503] Master 127.22.13.126:36117 was elected leader, sending a full tablet report...
I20260928 14:38:20.017972 29945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:20.015235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:33279 (local address 127.0.0.1:44204)
0928 14:38:20.015317 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:20.015336 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:20.015391 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:20.015556 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:20.015557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:20.015558 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:20.015610 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:20.015611 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.016242 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.016242 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:20.016994 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:20.016997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.017651 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.017651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:20.017723 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:20.017733 (+ 10us) client_negotiation.cc:769] Sending connection context
0928 14:38:20.017810 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:38:20.017866 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260928 14:38:20.017990 29937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:20.015316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44204 (local address 127.22.13.124:33279)
0928 14:38:20.015375 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:20.015377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:20.015388 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:38:20.015457 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:20.015458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:20.015477 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:20.015544 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:20.015666 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.016152 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.017074 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.017563 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.017634 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:20.017736 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:20.017822 (+ 86us) server_negotiation.cc:299] Negotiation successful
0928 14:38:20.017870 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 14:38:20.018368 29481 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260928 14:38:20.019270 29513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c3817afa240422eb13936fc7717656b. Latest consensus state: current_term: 1 leader_uuid: "8c3817afa240422eb13936fc7717656b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } } }
I20260928 14:38:20.019385 29513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [sys.catalog]: This master's current role is: LEARNER
I20260928 14:38:20.024423 29956 raft_consensus.cc:482] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:38:20.024554 29956 raft_consensus.cc:504] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } }
I20260928 14:38:20.024956 29956 leader_election.cc:288] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 456ec803412d4816b65345ebf7a8d8a5 (127.22.13.65:41419), ac34180e0f1c455da81fb0c4bc578d4b (127.22.13.67:38887)
I20260928 14:38:20.029132 29793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:20.025202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:41419 (local address 127.22.13.66:49345)
0928 14:38:20.025337 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:20.025348 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:20.025439 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:20.025649 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:20.025650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:20.025651 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:20.025698 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:20.025699 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.026589 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.026590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:20.027985 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:20.027989 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.028782 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.028783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:20.028868 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:20.028873 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:38:20.028938 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 14:38:20.029013 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":139}
I20260928 14:38:20.029160 29946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:20.025263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:41169 (local address 127.22.13.67:38887)
0928 14:38:20.025321 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:20.025323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:20.025334 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:38:20.025364 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:20.025365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:20.025374 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:20.025456 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:20.025638 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.026478 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.028087 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.028678 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.028766 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:20.028775 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:20.029024 (+ 249us) server_negotiation.cc:299] Negotiation successful
0928 14:38:20.029082 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260928 14:38:20.029160 29944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:20.025279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:49345 (local address 127.22.13.65:41419)
0928 14:38:20.025329 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:20.025331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:20.025421 (+ 90us) server_negotiation.cc:407] Connection header received
0928 14:38:20.025522 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:20.025523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:20.025532 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:20.025626 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:20.025761 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.026503 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.028088 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.028690 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.028766 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:20.028775 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:20.029036 (+ 261us) server_negotiation.cc:299] Negotiation successful
0928 14:38:20.029091 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:38:20.029142 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:20.025191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:38887 (local address 127.22.13.66:41169)
0928 14:38:20.025246 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:20.025258 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:20.025304 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:20.025460 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:20.025461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:20.025462 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:20.025560 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:20.025563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.026589 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.026590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:20.027988 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:20.027989 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.028782 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.028783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:20.028868 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:20.028873 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:38:20.028938 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 14:38:20.029017 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":111}
I20260928 14:38:20.029541 29604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5c2a73f852547ac92fff707e9f09488" candidate_uuid: "d844e25249cd4df5a06f0884bf9850f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456ec803412d4816b65345ebf7a8d8a5" is_pre_election: true
I20260928 14:38:20.029547 29877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5c2a73f852547ac92fff707e9f09488" candidate_uuid: "d844e25249cd4df5a06f0884bf9850f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" is_pre_election: true
I20260928 14:38:20.029698 29877 raft_consensus.cc:2454] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d844e25249cd4df5a06f0884bf9850f5 in term 0.
I20260928 14:38:20.029698 29604 raft_consensus.cc:2454] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d844e25249cd4df5a06f0884bf9850f5 in term 0.
I20260928 14:38:20.030000 29676 leader_election.cc:302] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456ec803412d4816b65345ebf7a8d8a5, d844e25249cd4df5a06f0884bf9850f5; no voters:
I20260928 14:38:20.030172 29956 raft_consensus.cc:2781] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:38:20.030258 29956 raft_consensus.cc:482] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:38:20.030280 29956 raft_consensus.cc:3037] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:20.031085 29956 raft_consensus.cc:504] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } }
I20260928 14:38:20.031255 29956 leader_election.cc:288] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [CANDIDATE]: Term 1 election: Requested vote from peers 456ec803412d4816b65345ebf7a8d8a5 (127.22.13.65:41419), ac34180e0f1c455da81fb0c4bc578d4b (127.22.13.67:38887)
I20260928 14:38:20.031423 29877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5c2a73f852547ac92fff707e9f09488" candidate_uuid: "d844e25249cd4df5a06f0884bf9850f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac34180e0f1c455da81fb0c4bc578d4b"
I20260928 14:38:20.031424 29604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5c2a73f852547ac92fff707e9f09488" candidate_uuid: "d844e25249cd4df5a06f0884bf9850f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456ec803412d4816b65345ebf7a8d8a5"
I20260928 14:38:20.031517 29604 raft_consensus.cc:3037] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:20.031517 29877 raft_consensus.cc:3037] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:20.032456 29877 raft_consensus.cc:2454] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d844e25249cd4df5a06f0884bf9850f5 in term 1.
I20260928 14:38:20.032466 29604 raft_consensus.cc:2454] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d844e25249cd4df5a06f0884bf9850f5 in term 1.
I20260928 14:38:20.032666 29677 leader_election.cc:302] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac34180e0f1c455da81fb0c4bc578d4b, d844e25249cd4df5a06f0884bf9850f5; no voters:
I20260928 14:38:20.032768 29956 raft_consensus.cc:2781] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:20.032917 29956 raft_consensus.cc:686] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [term 1 LEADER]: Becoming Leader. State: Replica: d844e25249cd4df5a06f0884bf9850f5, State: Running, Role: LEADER
I20260928 14:38:20.033022 29956 consensus_queue.cc:237] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } }
I20260928 14:38:20.033897 28684 catalog_manager.cc:5495] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to d844e25249cd4df5a06f0884bf9850f5 (127.22.13.66). New cstate: current_term: 1 leader_uuid: "d844e25249cd4df5a06f0884bf9850f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } health_report { overall_health: HEALTHY } } }
I20260928 14:38:20.034533 29435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c3817afa240422eb13936fc7717656b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: NON_VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: true } } }
I20260928 14:38:20.034662 29435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:20.035494 29435 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b: attempting to promote NON_VOTER 1a54d60db2c04cc0888795eeca75ff4a to VOTER
I20260928 14:38:20.035631 29435 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } }
I20260928 14:38:20.036161 29293 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 FOLLOWER]: Refusing update from remote peer 8c3817afa240422eb13936fc7717656b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260928 14:38:20.036202 29480 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 1 LEARNER]: Refusing update from remote peer 8c3817afa240422eb13936fc7717656b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260928 14:38:20.036383 29245 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260928 14:38:20.036485 29434 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260928 14:38:20.037178 29244 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 1a54d60db2c04cc0888795eeca75ff4a (127.22.13.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } } }
I20260928 14:38:20.037364 29293 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 1a54d60db2c04cc0888795eeca75ff4a (127.22.13.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } } }
I20260928 14:38:20.037319 29480 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 1a54d60db2c04cc0888795eeca75ff4a (127.22.13.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } } }
I20260928 14:38:20.038107 29244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8c3817afa240422eb13936fc7717656b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } } }
I20260928 14:38:20.038192 29244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:20.038236 29434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c3817afa240422eb13936fc7717656b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } } }
I20260928 14:38:20.038283 29434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:20.038158 29435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8c3817afa240422eb13936fc7717656b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } } }
I20260928 14:38:20.038337 29435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:20.038360 29433 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c3817afa240422eb13936fc7717656b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } } }
I20260928 14:38:20.038441 29433 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:38:20.038407 29513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8c3817afa240422eb13936fc7717656b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } } }
I20260928 14:38:20.038511 29513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:38:20.044247 29947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:20.041640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60002 (local address 127.22.13.66:36965)
0928 14:38:20.041690 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:20.041692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:20.041703 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:38:20.041795 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:20.041796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:20.041818 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:20.041921 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:38:20.042091 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.042568 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.043372 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.043586 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.043676 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:20.043883 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:38:20.044004 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:38:20.044106 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:20.044130 (+ 24us) server_negotiation.cc:299] Negotiation successful
0928 14:38:20.044156 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:38:20.046130 29604 raft_consensus.cc:1261] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5 [term 1 FOLLOWER]: Refusing update from remote peer d844e25249cd4df5a06f0884bf9850f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:20.046144 29877 raft_consensus.cc:1261] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b [term 1 FOLLOWER]: Refusing update from remote peer d844e25249cd4df5a06f0884bf9850f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:20.046541 29956 consensus_queue.cc:1035] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:20.046677 29960 consensus_queue.cc:1035] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:20.049556 29963 mvcc.cc:205] Tried to move back new op lower bound from 7334323404986114048 to 7334323404936024064. Current Snapshot: MvccSnapshot[applied={T|T < 7334323404986114048}]
I20260928 14:38:20.051445 29965 mvcc.cc:205] Tried to move back new op lower bound from 7334323404986114048 to 7334323404936024064. Current Snapshot: MvccSnapshot[applied={T|T < 7334323404986114048}]
I20260928 14:38:20.051575 29964 mvcc.cc:205] Tried to move back new op lower bound from 7334323404986114048 to 7334323404936024064. Current Snapshot: MvccSnapshot[applied={T|T < 7334323404986114048 or (T in {7334323404986114048})}]
I20260928 14:38:20.052570 28684 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:20.052656 28684 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:20.054121 28684 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c3817afa240422eb13936fc7717656b: Sending DeleteTablet for 3 replicas of tablet e5c2a73f852547ac92fff707e9f09488
I20260928 14:38:20.054464 29584 tablet_service.cc:1514] Processing DeleteTablet for tablet e5c2a73f852547ac92fff707e9f09488 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:20 UTC) from {username='oryx', principal='oryx/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:52544
I20260928 14:38:20.054472 29721 tablet_service.cc:1514] Processing DeleteTablet for tablet e5c2a73f852547ac92fff707e9f09488 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:20 UTC) from {username='oryx', principal='oryx/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:59998
I20260928 14:38:20.054582 29857 tablet_service.cc:1514] Processing DeleteTablet for tablet e5c2a73f852547ac92fff707e9f09488 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:20 UTC) from {username='oryx', principal='oryx/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:52098
I20260928 14:38:20.054705 29973 tablet_replica.cc:330] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5: stopping tablet replica
I20260928 14:38:20.054809 29973 raft_consensus.cc:2229] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:38:20.054839 29972 tablet_replica.cc:330] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5: stopping tablet replica
I20260928 14:38:20.054929 29973 raft_consensus.cc:2258] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:20.055050 29972 raft_consensus.cc:2229] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:20.055202 29972 raft_consensus.cc:2258] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:20.055408 29973 ts_tablet_manager.cc:1905] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:20.055712 29972 ts_tablet_manager.cc:1905] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:20.056030 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 28653
I20260928 14:38:20.057566 29973 ts_tablet_manager.cc:1918] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:38:20.057641 29973 log.cc:1196] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/e5c2a73f852547ac92fff707e9f09488
I20260928 14:38:20.057927 29973 ts_tablet_manager.cc:1939] T e5c2a73f852547ac92fff707e9f09488 P d844e25249cd4df5a06f0884bf9850f5: Deleting consensus metadata
I20260928 14:38:20.058492 29972 ts_tablet_manager.cc:1918] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:38:20.058578 29972 log.cc:1196] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-0/wal/wals/e5c2a73f852547ac92fff707e9f09488
I20260928 14:38:20.058893 29972 ts_tablet_manager.cc:1939] T e5c2a73f852547ac92fff707e9f09488 P 456ec803412d4816b65345ebf7a8d8a5: Deleting consensus metadata
I20260928 14:38:20.062361 29974 tablet_replica.cc:330] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b: stopping tablet replica
I20260928 14:38:20.062575 29974 raft_consensus.cc:2229] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:20.062765 29974 raft_consensus.cc:2258] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260928 14:38:20.063241 29674 connection.cc:535] server connection from 127.0.0.1:59998 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 14:38:20.063385 29974 ts_tablet_manager.cc:1905] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260928 14:38:20.063709 29540 connection.cc:535] server connection from 127.0.0.1:52544 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 14:38:20.065344 29974 ts_tablet_manager.cc:1918] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:38:20.065476 29974 log.cc:1196] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/e5c2a73f852547ac92fff707e9f09488
I20260928 14:38:20.065814 29974 ts_tablet_manager.cc:1939] T e5c2a73f852547ac92fff707e9f09488 P ac34180e0f1c455da81fb0c4bc578d4b: Deleting consensus metadata
W20260928 14:38:20.066300 29811 connection.cc:441] server connection from 127.0.0.1:52098 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52098 (request call id 1) could send its response
I20260928 14:38:20.070726 29937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:20.055504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44214 (local address 127.22.13.124:33279)
0928 14:38:20.055547 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:20.055549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:20.062231 (+ 6682us) server_negotiation.cc:407] Connection header received
0928 14:38:20.062275 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:20.062276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:20.062288 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:20.062373 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:20.063205 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.064004 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.065122 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.065426 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.065465 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:20.067225 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:20.067226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:20.067227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:20.067234 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:20.068931 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:20.069330 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:20.069331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:20.069332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:20.069384 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:20.069569 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:20.069570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:20.069570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:20.070001 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:20.070083 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:20.070549 (+ 466us) server_negotiation.cc:299] Negotiation successful
0928 14:38:20.070627 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:38:20.074074 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:20.065690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44218 (local address 127.22.13.124:33279)
0928 14:38:20.065749 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:20.065751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:20.066241 (+ 490us) server_negotiation.cc:407] Connection header received
0928 14:38:20.066467 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:20.066469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:20.066482 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:20.066536 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:20.067286 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.068022 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.069027 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.069322 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.069361 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:20.071198 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:20.071199 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:20.071199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:20.071205 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:20.072736 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:20.073153 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:20.073154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:20.073155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:20.073209 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:20.073435 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:20.073436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:20.073437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:20.073571 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:20.073647 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:20.073864 (+ 217us) server_negotiation.cc:299] Negotiation successful
0928 14:38:20.073942 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:38:20.074167 29939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:20.066634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44400 (local address 127.22.13.125:43783)
0928 14:38:20.066687 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:20.066689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:20.067026 (+ 337us) server_negotiation.cc:407] Connection header received
0928 14:38:20.067140 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:20.067141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:20.067156 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:20.067189 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:20.067799 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.068498 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.069455 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:20.069779 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:20.069821 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:20.071187 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:20.071188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:20.071188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:20.071195 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:20.072879 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:20.073321 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:20.073323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:20.073323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:20.073375 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:20.073589 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:20.073590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:20.073590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:20.073731 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:20.073826 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:20.073987 (+ 161us) server_negotiation.cc:299] Negotiation successful
0928 14:38:20.074070 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:38:21.051857 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:21.051498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.65:56241)
0928 14:38:21.051716 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:21.051799 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":64,"threads_started":1}
I20260928 14:38:21.052055 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:21.051823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.66:56979)
0928 14:38:21.051934 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:21.052009 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":53,"threads_started":1}
I20260928 14:38:21.052083 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:21.051728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.67:56021)
0928 14:38:21.051982 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:21.052046 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":48,"threads_started":1}
W20260928 14:38:21.052124 29651 heartbeater.cc:644] Failed to heartbeat to 127.22.13.126:36117 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
W20260928 14:38:21.052209 29788 heartbeater.cc:644] Failed to heartbeat to 127.22.13.126:36117 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
W20260928 14:38:21.052220 29924 heartbeater.cc:644] Failed to heartbeat to 127.22.13.126:36117 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
I20260928 14:38:21.554502 29986 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c3817afa240422eb13936fc7717656b)
I20260928 14:38:21.554636 29986 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } }
I20260928 14:38:21.554980 29986 leader_election.cc:288] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3817afa240422eb13936fc7717656b (127.22.13.126:36117), 1a54d60db2c04cc0888795eeca75ff4a (127.22.13.124:33279)
I20260928 14:38:21.554980 29987 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c3817afa240422eb13936fc7717656b)
I20260928 14:38:21.555191 29987 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } }
I20260928 14:38:21.555387 29988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:21.555138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:46712)
0928 14:38:21.555278 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:21.555345 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
I20260928 14:38:21.555454 29987 leader_election.cc:288] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3817afa240422eb13936fc7717656b (127.22.13.126:36117), ad5654dbe1244b828d6054a73f683c75 (127.22.13.125:43783)
W20260928 14:38:21.555509 29268 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
I20260928 14:38:21.555755 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:21.555517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:46728)
0928 14:38:21.555672 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:21.555717 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
W20260928 14:38:21.555867 29456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
I20260928 14:38:21.556010 29988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:21.555918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:46736)
0928 14:38:21.555953 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:21.555974 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260928 14:38:21.556089 29268 leader_election.cc:334] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8c3817afa240422eb13936fc7717656b (127.22.13.126:36117): Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
I20260928 14:38:21.556255 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:21.556168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:46746)
0928 14:38:21.556201 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:21.556224 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260928 14:38:21.556345 29456 leader_election.cc:334] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8c3817afa240422eb13936fc7717656b (127.22.13.126:36117): Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
I20260928 14:38:21.558897 29990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:21.555311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44232 (local address 127.22.13.124:33279)
0928 14:38:21.555394 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:21.555396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:21.555725 (+ 329us) server_negotiation.cc:407] Connection header received
0928 14:38:21.555752 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:21.555753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:21.555763 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:21.555983 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:21.556307 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:21.557053 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:21.558170 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:21.558586 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:21.558637 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:21.558654 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:21.558781 (+ 127us) server_negotiation.cc:299] Negotiation successful
0928 14:38:21.558814 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61,"thread_start_us":37,"threads_started":1}
I20260928 14:38:21.558892 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:21.555190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:33279 (local address 127.0.0.1:44232)
0928 14:38:21.555535 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:21.555546 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:21.555669 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:21.556034 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:21.556035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:21.556037 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:21.556224 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:21.556227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:21.557152 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:21.557153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:21.558093 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:21.558095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:21.558648 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:21.558649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:21.558704 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:21.558708 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:38:21.558771 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 14:38:21.558813 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":51,"threads_started":1}
I20260928 14:38:21.559120 29993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:21.555691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:43783 (local address 127.0.0.1:44414)
0928 14:38:21.555783 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:21.555792 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:21.555846 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:21.556064 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:21.556065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:21.556068 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:21.556131 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:21.556133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:21.557312 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:21.557313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:21.558178 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:21.558179 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:21.558860 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:21.558860 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:21.558919 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:21.558923 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:38:21.559025 (+ 102us) client_negotiation.cc:240] Negotiation successful
0928 14:38:21.559066 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":42,"threads_started":1}
I20260928 14:38:21.559166 29480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad5654dbe1244b828d6054a73f683c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1a54d60db2c04cc0888795eeca75ff4a" is_pre_election: true
I20260928 14:38:21.559288 29994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:21.555688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.22.13.125:43783)
0928 14:38:21.555933 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:21.555934 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:21.555941 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:38:21.555964 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:21.555965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:21.555974 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:21.556216 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:21.556243 (+ 27us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:21.557237 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:21.558254 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:21.558789 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:21.558846 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:21.558854 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:21.559192 (+ 338us) server_negotiation.cc:299] Negotiation successful
0928 14:38:21.559238 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":74,"threads_started":1}
I20260928 14:38:21.559290 29480 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5654dbe1244b828d6054a73f683c75 in term 1.
I20260928 14:38:21.559456 29270 leader_election.cc:302] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1a54d60db2c04cc0888795eeca75ff4a, ad5654dbe1244b828d6054a73f683c75; no voters: 8c3817afa240422eb13936fc7717656b
I20260928 14:38:21.559441 29293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a54d60db2c04cc0888795eeca75ff4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ad5654dbe1244b828d6054a73f683c75" is_pre_election: true
I20260928 14:38:21.559585 29293 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a54d60db2c04cc0888795eeca75ff4a in term 1.
I20260928 14:38:21.559659 29986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260928 14:38:21.559702 29986 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c3817afa240422eb13936fc7717656b)
I20260928 14:38:21.559725 29986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:38:21.559739 29455 leader_election.cc:302] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1a54d60db2c04cc0888795eeca75ff4a, ad5654dbe1244b828d6054a73f683c75; no voters: 8c3817afa240422eb13936fc7717656b
I20260928 14:38:21.559834 29987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260928 14:38:21.559880 29987 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8c3817afa240422eb13936fc7717656b)
I20260928 14:38:21.559898 29987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:38:21.560668 29986 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } }
I20260928 14:38:21.560745 29987 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } }
I20260928 14:38:21.560834 29986 leader_election.cc:288] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [CANDIDATE]: Term 2 election: Requested vote from peers 8c3817afa240422eb13936fc7717656b (127.22.13.126:36117), 1a54d60db2c04cc0888795eeca75ff4a (127.22.13.124:33279)
I20260928 14:38:21.560851 29987 leader_election.cc:288] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [CANDIDATE]: Term 2 election: Requested vote from peers 8c3817afa240422eb13936fc7717656b (127.22.13.126:36117), ad5654dbe1244b828d6054a73f683c75 (127.22.13.125:43783)
I20260928 14:38:21.560974 29480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad5654dbe1244b828d6054a73f683c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1a54d60db2c04cc0888795eeca75ff4a"
I20260928 14:38:21.561070 29480 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ad5654dbe1244b828d6054a73f683c75 in current term 2: Already voted for candidate 1a54d60db2c04cc0888795eeca75ff4a in this term.
I20260928 14:38:21.561105 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:21.561006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:46766)
0928 14:38:21.561044 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:21.561078 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260928 14:38:21.561198 29293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a54d60db2c04cc0888795eeca75ff4a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ad5654dbe1244b828d6054a73f683c75"
I20260928 14:38:21.561285 29293 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1a54d60db2c04cc0888795eeca75ff4a in current term 2: Already voted for candidate ad5654dbe1244b828d6054a73f683c75 in this term.
I20260928 14:38:21.561550 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:21.561486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:46792)
0928 14:38:21.561510 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:21.561534 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
I20260928 14:38:21.561599 29993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:21.561139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:46778)
0928 14:38:21.561529 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:21.561561 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375}
W20260928 14:38:21.561625 29268 leader_election.cc:334] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8c3817afa240422eb13936fc7717656b (127.22.13.126:36117): Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
I20260928 14:38:21.561659 29268 leader_election.cc:302] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad5654dbe1244b828d6054a73f683c75; no voters: 1a54d60db2c04cc0888795eeca75ff4a, 8c3817afa240422eb13936fc7717656b
I20260928 14:38:21.561762 29986 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260928 14:38:21.561913 29993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:21.561837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:46806)
0928 14:38:21.561869 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:21.561893 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260928 14:38:21.562031 29456 leader_election.cc:334] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8c3817afa240422eb13936fc7717656b (127.22.13.126:36117): Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
I20260928 14:38:21.562098 29456 leader_election.cc:302] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a54d60db2c04cc0888795eeca75ff4a; no voters: 8c3817afa240422eb13936fc7717656b, ad5654dbe1244b828d6054a73f683c75
I20260928 14:38:21.562173 29987 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260928 14:38:22.052719 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:22.052493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.65:43047)
0928 14:38:22.052620 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:22.052662 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260928 14:38:22.052742 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:22.052538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.67:57209)
0928 14:38:22.052641 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:22.052693 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":57,"threads_started":1}
I20260928 14:38:22.052770 29996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:22.052493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.66:48569)
0928 14:38:22.052696 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:22.052728 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":62,"threads_started":1}
I20260928 14:38:23.053676 29999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.053230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.65:59841)
0928 14:38:23.053570 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.053605 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":64,"threads_started":1}
I20260928 14:38:23.053767 30001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.053347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.66:44443)
0928 14:38:23.053674 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.053717 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":54,"threads_started":1}
I20260928 14:38:23.053967 30000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.053238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.67:39497)
0928 14:38:23.053489 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.053537 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":60,"threads_started":1}
I20260928 14:38:23.535962 30002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:38:23.536063 30002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } }
I20260928 14:38:23.536257 30002 leader_election.cc:288] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8c3817afa240422eb13936fc7717656b (127.22.13.126:36117), ad5654dbe1244b828d6054a73f683c75 (127.22.13.125:43783)
I20260928 14:38:23.536494 29293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a54d60db2c04cc0888795eeca75ff4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ad5654dbe1244b828d6054a73f683c75" is_pre_election: true
I20260928 14:38:23.536594 29293 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a54d60db2c04cc0888795eeca75ff4a in term 2.
I20260928 14:38:23.536598 30003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.536397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:47046)
0928 14:38:23.536503 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.536558 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":62,"threads_started":1}
I20260928 14:38:23.536752 29455 leader_election.cc:302] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a54d60db2c04cc0888795eeca75ff4a, ad5654dbe1244b828d6054a73f683c75; no voters:
I20260928 14:38:23.536860 30002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260928 14:38:23.536916 30002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:38:23.536942 30002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 2 FOLLOWER]: Advancing to term 3
I20260928 14:38:23.537068 30003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.536974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:47062)
0928 14:38:23.537009 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.537037 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260928 14:38:23.537164 29456 leader_election.cc:334] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8c3817afa240422eb13936fc7717656b (127.22.13.126:36117): Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
I20260928 14:38:23.537727 30002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } }
I20260928 14:38:23.537863 30002 leader_election.cc:288] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [CANDIDATE]: Term 3 election: Requested vote from peers 8c3817afa240422eb13936fc7717656b (127.22.13.126:36117), ad5654dbe1244b828d6054a73f683c75 (127.22.13.125:43783)
I20260928 14:38:23.538061 30003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.537961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:47078)
0928 14:38:23.537997 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.538027 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260928 14:38:23.538055 29293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a54d60db2c04cc0888795eeca75ff4a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ad5654dbe1244b828d6054a73f683c75"
I20260928 14:38:23.538128 29293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 2 FOLLOWER]: Advancing to term 3
I20260928 14:38:23.538425 30003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.538333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:47092)
0928 14:38:23.538382 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.538406 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260928 14:38:23.538568 29456 leader_election.cc:334] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8c3817afa240422eb13936fc7717656b (127.22.13.126:36117): Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
I20260928 14:38:23.538791 29293 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a54d60db2c04cc0888795eeca75ff4a in term 3.
I20260928 14:38:23.538937 29455 leader_election.cc:302] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1a54d60db2c04cc0888795eeca75ff4a, ad5654dbe1244b828d6054a73f683c75; no voters: 8c3817afa240422eb13936fc7717656b
I20260928 14:38:23.539073 30002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 3 FOLLOWER]: Leader election won for term 3
I20260928 14:38:23.539215 30002 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [term 3 LEADER]: Becoming Leader. State: Replica: 1a54d60db2c04cc0888795eeca75ff4a, State: Running, Role: LEADER
I20260928 14:38:23.539304 30002 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } }
I20260928 14:38:23.539708 30005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a54d60db2c04cc0888795eeca75ff4a. Latest consensus state: current_term: 3 leader_uuid: "1a54d60db2c04cc0888795eeca75ff4a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } } }
I20260928 14:38:23.539820 30005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:23.539996 30007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:23.540444 30007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:23.540506 30007 catalog_manager.cc:1259] Loaded cluster ID: a0d4d515ab95487f91125cdd920f6af3
I20260928 14:38:23.540524 30007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:38:23.540686 30007 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:23.540737 30007 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: Loaded TSK: 0
I20260928 14:38:23.540836 30007 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:38:23.570307 29471 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:38:23.571902 29471 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:38:23.572926 30003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.572782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:47114)
0928 14:38:23.572852 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.572893 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 14:38:23.572929 29293 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [term 3 FOLLOWER]: Refusing update from remote peer 1a54d60db2c04cc0888795eeca75ff4a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260928 14:38:23.573161 30002 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260928 14:38:23.573395 30003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.573287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.0.0.1:47128)
0928 14:38:23.573341 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.573370 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260928 14:38:23.573488 29456 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a -> Peer 8c3817afa240422eb13936fc7717656b (127.22.13.126:36117): Couldn't send request to peer 8c3817afa240422eb13936fc7717656b. Status: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260928 14:38:23.573822 30009 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a54d60db2c04cc0888795eeca75ff4a. Latest consensus state: current_term: 3 leader_uuid: "1a54d60db2c04cc0888795eeca75ff4a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } } }
I20260928 14:38:23.573962 30009 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:38:23.574385 30002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "1a54d60db2c04cc0888795eeca75ff4a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } } }
I20260928 14:38:23.574478 30002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:23.574589 30009 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "1a54d60db2c04cc0888795eeca75ff4a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8c3817afa240422eb13936fc7717656b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36117 } } peers { permanent_uuid: "ad5654dbe1244b828d6054a73f683c75" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 43783 } attrs { promote: false } } peers { permanent_uuid: "1a54d60db2c04cc0888795eeca75ff4a" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 33279 } attrs { promote: false } } }
I20260928 14:38:23.574685 30009 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad5654dbe1244b828d6054a73f683c75 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:38:23.579522 30015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.576448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:36965 (local address 127.0.0.1:60006)
0928 14:38:23.576579 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.576590 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:23.576641 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:23.576973 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:23.576974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:23.576976 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:23.577023 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:23.577025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.577694 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.577695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:23.578500 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:23.578502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.579288 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.579289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:23.579353 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:23.579357 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:38:23.579401 (+ 44us) client_negotiation.cc:240] Negotiation successful
0928 14:38:23.579441 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":87,"threads_started":1}
I20260928 14:38:23.579751 30003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.576354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:41419 (local address 127.0.0.1:52546)
0928 14:38:23.576384 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.576396 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:23.576444 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:23.576742 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:23.576743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:23.576745 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:23.576805 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:23.576807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.577599 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.577600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:23.578802 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:23.578804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.579486 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.579486 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:23.579549 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:23.579553 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:38:23.579628 (+ 75us) client_negotiation.cc:240] Negotiation successful
0928 14:38:23.579682 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260928 14:38:23.579777 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.576666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60006 (local address 127.22.13.66:36965)
0928 14:38:23.576825 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:23.576827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:23.576834 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:38:23.576871 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:23.576872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:23.576882 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:23.576969 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:23.577074 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.577596 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.578599 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.579173 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.579246 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:23.579259 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:23.579663 (+ 404us) server_negotiation.cc:299] Negotiation successful
0928 14:38:23.579713 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":60,"threads_started":1}
I20260928 14:38:23.579823 30014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.576432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52546 (local address 127.22.13.65:41419)
0928 14:38:23.576615 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:23.576618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:23.576627 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:38:23.576654 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:23.576654 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:23.576665 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:23.576731 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:23.576885 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.577503 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.578885 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.579392 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.579475 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:23.579487 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:23.579699 (+ 212us) server_negotiation.cc:299] Negotiation successful
0928 14:38:23.579742 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":53,"threads_started":1}
I20260928 14:38:23.579988 29721 tablet_service.cc:1467] Processing CreateTablet for tablet 0a81860cc3a54577a0ebaf6346736f49 (DEFAULT_TABLE table=test-table [id=a5ec7f8dc0e74db4a4e2467f8ec81cab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:23.580129 29721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a81860cc3a54577a0ebaf6346736f49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:23.581141 30016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.576449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:38887 (local address 127.0.0.1:52100)
0928 14:38:23.576717 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.576726 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:23.576761 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:23.577737 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:23.577737 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:23.577739 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:23.577771 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:23.577772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.578688 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.578689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:23.580271 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:23.580273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.580975 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.580975 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:23.581016 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:23.581019 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:38:23.581054 (+ 35us) client_negotiation.cc:240] Negotiation successful
0928 14:38:23.581085 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":89,"threads_started":1}
I20260928 14:38:23.581183 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.576572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52100 (local address 127.22.13.67:38887)
0928 14:38:23.576808 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:23.576810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:23.576817 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:38:23.577621 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:23.577622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:23.577636 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:23.577694 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:23.577839 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.578614 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.580354 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.580884 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.580963 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:23.580979 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:23.581069 (+ 90us) server_negotiation.cc:299] Negotiation successful
0928 14:38:23.581118 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":47,"threads_started":1}
I20260928 14:38:23.581661 29857 tablet_service.cc:1467] Processing CreateTablet for tablet 0a81860cc3a54577a0ebaf6346736f49 (DEFAULT_TABLE table=test-table [id=a5ec7f8dc0e74db4a4e2467f8ec81cab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:23.581804 29857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a81860cc3a54577a0ebaf6346736f49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:23.581807 30019 tablet_bootstrap.cc:492] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5: Bootstrap starting.
I20260928 14:38:23.582183 29584 tablet_service.cc:1467] Processing CreateTablet for tablet 0a81860cc3a54577a0ebaf6346736f49 (DEFAULT_TABLE table=test-table [id=a5ec7f8dc0e74db4a4e2467f8ec81cab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:23.582273 30019 tablet_bootstrap.cc:654] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:23.582309 29584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a81860cc3a54577a0ebaf6346736f49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:23.582909 30019 tablet_bootstrap.cc:492] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5: No bootstrap required, opened a new log
I20260928 14:38:23.582965 30019 ts_tablet_manager.cc:1397] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:23.583094 30019 raft_consensus.cc:348] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } }
I20260928 14:38:23.583194 30019 raft_consensus.cc:374] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:23.583223 30019 raft_consensus.cc:729] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d844e25249cd4df5a06f0884bf9850f5, State: Initialized, Role: FOLLOWER
I20260928 14:38:23.583264 30019 consensus_queue.cc:260] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } }
I20260928 14:38:23.583413 30019 ts_tablet_manager.cc:1428] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 14:38:23.583873 30024 tablet_bootstrap.cc:492] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5: Bootstrap starting.
I20260928 14:38:23.584131 30023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.583588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.66:53917)
0928 14:38:23.584056 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.584095 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":78,"threads_started":1}
I20260928 14:38:23.584383 29787 heartbeater.cc:503] Master 127.22.13.124:33279 was elected leader, sending a full tablet report...
I20260928 14:38:23.584491 30025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.584310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.66:59189)
0928 14:38:23.584422 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.584458 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":74,"threads_started":1}
I20260928 14:38:23.584486 30021 tablet_bootstrap.cc:492] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b: Bootstrap starting.
I20260928 14:38:23.584506 30024 tablet_bootstrap.cc:654] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:23.584919 30021 tablet_bootstrap.cc:654] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:23.585043 30024 tablet_bootstrap.cc:492] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5: No bootstrap required, opened a new log
I20260928 14:38:23.585096 30024 ts_tablet_manager.cc:1397] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:23.585238 30024 raft_consensus.cc:348] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } }
I20260928 14:38:23.585321 30024 raft_consensus.cc:374] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:23.585352 30024 raft_consensus.cc:729] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456ec803412d4816b65345ebf7a8d8a5, State: Initialized, Role: FOLLOWER
I20260928 14:38:23.585398 30024 consensus_queue.cc:260] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } }
I20260928 14:38:23.585495 30021 tablet_bootstrap.cc:492] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b: No bootstrap required, opened a new log
I20260928 14:38:23.585559 30024 ts_tablet_manager.cc:1428] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:23.585551 30021 ts_tablet_manager.cc:1397] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:23.585790 30021 raft_consensus.cc:348] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } }
I20260928 14:38:23.585876 30021 raft_consensus.cc:374] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:23.585899 30021 raft_consensus.cc:729] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac34180e0f1c455da81fb0c4bc578d4b, State: Initialized, Role: FOLLOWER
I20260928 14:38:23.585970 30021 consensus_queue.cc:260] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } }
I20260928 14:38:23.586045 29650 heartbeater.cc:503] Master 127.22.13.124:33279 was elected leader, sending a full tablet report...
I20260928 14:38:23.586129 30021 ts_tablet_manager.cc:1428] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:23.586562 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.586324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.67:48367)
0928 14:38:23.586493 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.586527 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":70,"threads_started":1}
I20260928 14:38:23.586727 29923 heartbeater.cc:503] Master 127.22.13.124:33279 was elected leader, sending a full tablet report...
I20260928 14:38:23.586856 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.585761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.65:52335)
0928 14:38:23.586772 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.586808 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":986,"thread_start_us":69,"threads_started":1}
I20260928 14:38:23.586895 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.586818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.67:54179)
0928 14:38:23.586851 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.586871 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260928 14:38:23.587126 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.587041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.65:48911)
0928 14:38:23.587071 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.587093 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260928 14:38:23.644658 30030 raft_consensus.cc:482] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:38:23.644753 30030 raft_consensus.cc:504] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } }
I20260928 14:38:23.645118 30030 leader_election.cc:288] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 456ec803412d4816b65345ebf7a8d8a5 (127.22.13.65:41419), d844e25249cd4df5a06f0884bf9850f5 (127.22.13.66:36965)
I20260928 14:38:23.649355 30014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.645422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:48653 (local address 127.22.13.65:41419)
0928 14:38:23.645483 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:23.645485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:23.645500 (+ 15us) server_negotiation.cc:407] Connection header received
0928 14:38:23.645596 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:23.645597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:23.645607 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:23.645696 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:23.645876 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.646687 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.648393 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.648959 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.649044 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:23.649066 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:23.649239 (+ 173us) server_negotiation.cc:299] Negotiation successful
0928 14:38:23.649279 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:38:23.649355 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.645572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:46925 (local address 127.22.13.66:36965)
0928 14:38:23.645615 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:23.645617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:23.645699 (+ 82us) server_negotiation.cc:407] Connection header received
0928 14:38:23.645731 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:23.645732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:23.645740 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:23.645792 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:23.645979 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.646687 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.648393 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.648959 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.649044 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:23.649056 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:23.649239 (+ 183us) server_negotiation.cc:299] Negotiation successful
0928 14:38:23.649280 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:38:23.649374 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.645345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:41419 (local address 127.22.13.67:48653)
0928 14:38:23.645438 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.645450 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:23.645507 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:23.645692 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:23.645693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:23.645696 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:23.645798 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:23.645801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.646793 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.646794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:23.648277 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:23.648280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.649066 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.649067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:23.649150 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:23.649154 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:38:23.649215 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 14:38:23.649275 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":62}
I20260928 14:38:23.649683 30032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.645457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:36965 (local address 127.22.13.67:46925)
0928 14:38:23.645606 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.645614 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:23.645688 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:23.645820 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:23.645821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:23.645822 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:23.645877 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:23.645878 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.646793 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.646794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:23.648277 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:23.648280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.649086 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.649086 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:23.649155 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:23.649158 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:38:23.649215 (+ 57us) client_negotiation.cc:240] Negotiation successful
0928 14:38:23.649275 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":118,"thread_start_us":115,"threads_started":1}
I20260928 14:38:23.649667 29604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a81860cc3a54577a0ebaf6346736f49" candidate_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456ec803412d4816b65345ebf7a8d8a5" is_pre_election: true
I20260928 14:38:23.649775 29604 raft_consensus.cc:2454] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac34180e0f1c455da81fb0c4bc578d4b in term 0.
I20260928 14:38:23.650022 29812 leader_election.cc:302] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456ec803412d4816b65345ebf7a8d8a5, ac34180e0f1c455da81fb0c4bc578d4b; no voters:
I20260928 14:38:23.650213 30022 raft_consensus.cc:482] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:38:23.650220 30030 raft_consensus.cc:2781] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:38:23.650269 30030 raft_consensus.cc:482] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:38:23.650286 30030 raft_consensus.cc:3037] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:23.650280 30022 raft_consensus.cc:504] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } }
I20260928 14:38:23.650322 29741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a81860cc3a54577a0ebaf6346736f49" candidate_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844e25249cd4df5a06f0884bf9850f5" is_pre_election: true
I20260928 14:38:23.650427 30022 leader_election.cc:288] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 456ec803412d4816b65345ebf7a8d8a5 (127.22.13.65:41419), ac34180e0f1c455da81fb0c4bc578d4b (127.22.13.67:38887)
I20260928 14:38:23.650435 29741 raft_consensus.cc:2454] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac34180e0f1c455da81fb0c4bc578d4b in term 0.
I20260928 14:38:23.650571 29604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a81860cc3a54577a0ebaf6346736f49" candidate_uuid: "d844e25249cd4df5a06f0884bf9850f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456ec803412d4816b65345ebf7a8d8a5" is_pre_election: true
I20260928 14:38:23.650573 29877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a81860cc3a54577a0ebaf6346736f49" candidate_uuid: "d844e25249cd4df5a06f0884bf9850f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" is_pre_election: true
I20260928 14:38:23.650645 29604 raft_consensus.cc:2454] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d844e25249cd4df5a06f0884bf9850f5 in term 0.
I20260928 14:38:23.650801 29676 leader_election.cc:302] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 456ec803412d4816b65345ebf7a8d8a5, d844e25249cd4df5a06f0884bf9850f5; no voters:
I20260928 14:38:23.650885 30022 raft_consensus.cc:2781] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:38:23.650914 30022 raft_consensus.cc:482] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:38:23.650940 30022 raft_consensus.cc:3037] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:23.651520 30030 raft_consensus.cc:504] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } }
I20260928 14:38:23.651682 30030 leader_election.cc:288] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [CANDIDATE]: Term 1 election: Requested vote from peers 456ec803412d4816b65345ebf7a8d8a5 (127.22.13.65:41419), d844e25249cd4df5a06f0884bf9850f5 (127.22.13.66:36965)
I20260928 14:38:23.651701 29877 raft_consensus.cc:2379] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d844e25249cd4df5a06f0884bf9850f5 in current term 1: Already voted for candidate ac34180e0f1c455da81fb0c4bc578d4b in this term.
I20260928 14:38:23.651801 30022 raft_consensus.cc:504] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } }
I20260928 14:38:23.651835 29741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a81860cc3a54577a0ebaf6346736f49" candidate_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d844e25249cd4df5a06f0884bf9850f5"
I20260928 14:38:23.651865 29604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a81860cc3a54577a0ebaf6346736f49" candidate_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456ec803412d4816b65345ebf7a8d8a5"
I20260928 14:38:23.651937 29604 raft_consensus.cc:3037] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:23.651937 30022 leader_election.cc:288] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [CANDIDATE]: Term 1 election: Requested vote from peers 456ec803412d4816b65345ebf7a8d8a5 (127.22.13.65:41419), ac34180e0f1c455da81fb0c4bc578d4b (127.22.13.67:38887)
I20260928 14:38:23.651959 29741 raft_consensus.cc:2379] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ac34180e0f1c455da81fb0c4bc578d4b in current term 1: Already voted for candidate d844e25249cd4df5a06f0884bf9850f5 in this term.
I20260928 14:38:23.652164 29603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a81860cc3a54577a0ebaf6346736f49" candidate_uuid: "d844e25249cd4df5a06f0884bf9850f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "456ec803412d4816b65345ebf7a8d8a5"
I20260928 14:38:23.652644 29604 raft_consensus.cc:2454] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac34180e0f1c455da81fb0c4bc578d4b in term 1.
I20260928 14:38:23.652815 29812 leader_election.cc:302] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 456ec803412d4816b65345ebf7a8d8a5, ac34180e0f1c455da81fb0c4bc578d4b; no voters: d844e25249cd4df5a06f0884bf9850f5
I20260928 14:38:23.652911 29877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a81860cc3a54577a0ebaf6346736f49" candidate_uuid: "d844e25249cd4df5a06f0884bf9850f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac34180e0f1c455da81fb0c4bc578d4b"
I20260928 14:38:23.652987 29877 raft_consensus.cc:2379] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d844e25249cd4df5a06f0884bf9850f5 in current term 1: Already voted for candidate ac34180e0f1c455da81fb0c4bc578d4b in this term.
I20260928 14:38:23.653059 30030 raft_consensus.cc:2781] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:23.653118 29677 leader_election.cc:302] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d844e25249cd4df5a06f0884bf9850f5; no voters: 456ec803412d4816b65345ebf7a8d8a5, ac34180e0f1c455da81fb0c4bc578d4b
I20260928 14:38:23.653206 30030 raft_consensus.cc:686] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [term 1 LEADER]: Becoming Leader. State: Replica: ac34180e0f1c455da81fb0c4bc578d4b, State: Running, Role: LEADER
I20260928 14:38:23.653206 30022 raft_consensus.cc:2726] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260928 14:38:23.653278 30030 consensus_queue.cc:237] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } }
I20260928 14:38:23.653891 30032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.653762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.67:56425)
0928 14:38:23.653826 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.653856 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260928 14:38:23.654049 29471 catalog_manager.cc:5495] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b reported cstate change: term changed from 0 to 1, leader changed from <none> to ac34180e0f1c455da81fb0c4bc578d4b (127.22.13.67). New cstate: current_term: 1 leader_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac34180e0f1c455da81fb0c4bc578d4b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38887 } health_report { overall_health: HEALTHY } } }
I20260928 14:38:23.672353 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.670148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52112 (local address 127.22.13.67:38887)
0928 14:38:23.670191 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:23.670193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:23.670229 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:38:23.670271 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:23.670271 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:23.670289 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:23.670343 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:38:23.670467 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.670960 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.671657 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:23.671836 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:23.671886 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:23.672034 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:38:23.672147 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:38:23.672236 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:23.672271 (+ 35us) server_negotiation.cc:299] Negotiation successful
0928 14:38:23.672290 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:38:23.673568 29604 raft_consensus.cc:1261] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5 [term 1 FOLLOWER]: Refusing update from remote peer ac34180e0f1c455da81fb0c4bc578d4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:23.673648 29741 raft_consensus.cc:1261] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [term 1 FOLLOWER]: Refusing update from remote peer ac34180e0f1c455da81fb0c4bc578d4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:23.673839 30030 consensus_queue.cc:1035] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "456ec803412d4816b65345ebf7a8d8a5" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 41419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:23.674034 30033 consensus_queue.cc:1035] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d844e25249cd4df5a06f0884bf9850f5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:23.674947 30025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.674824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.66:48799)
0928 14:38:23.674867 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.674910 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260928 14:38:23.675285 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.674862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.65:33085)
0928 14:38:23.675223 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.675255 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328}
I20260928 14:38:23.676687 30032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.676449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.67:42753)
0928 14:38:23.676611 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.676653 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260928 14:38:23.677222 30025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.677114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.66:41103)
0928 14:38:23.677160 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.677192 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260928 14:38:23.677502 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:23.676618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36117 (local address 127.22.13.65:44199)
0928 14:38:23.677434 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.677465 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":792}
I20260928 14:38:23.678774 29470 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:23.678877 29470 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:23.680168 29470 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a54d60db2c04cc0888795eeca75ff4a: Sending DeleteTablet for 3 replicas of tablet 0a81860cc3a54577a0ebaf6346736f49
I20260928 14:38:23.680477 29857 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a81860cc3a54577a0ebaf6346736f49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:23 UTC) from {username='oryx', principal='oryx/127.22.13.124@KRBTEST.COM'} at 127.0.0.1:52100
I20260928 14:38:23.680629 29584 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a81860cc3a54577a0ebaf6346736f49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:23 UTC) from {username='oryx', principal='oryx/127.22.13.124@KRBTEST.COM'} at 127.0.0.1:52546
I20260928 14:38:23.680707 29721 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a81860cc3a54577a0ebaf6346736f49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:23 UTC) from {username='oryx', principal='oryx/127.22.13.124@KRBTEST.COM'} at 127.0.0.1:60006
I20260928 14:38:23.680841 30048 tablet_replica.cc:330] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5: stopping tablet replica
I20260928 14:38:23.680840 30047 tablet_replica.cc:330] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b: stopping tablet replica
I20260928 14:38:23.680907 30048 raft_consensus.cc:2229] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:23.680963 30048 raft_consensus.cc:2258] T 0a81860cc3a54577a0ebaf6346736f49 P 456ec803412d4816b65345ebf7a8d8a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:23.681037 30047 raft_consensus.cc:2229] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:38:23.681195 30047 raft_consensus.cc:2258] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:23.681332 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 29430
I20260928 14:38:23.682508 30049 tablet_replica.cc:330] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5: stopping tablet replica
I20260928 14:38:23.682595 30049 raft_consensus.cc:2229] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:23.682662 30049 raft_consensus.cc:2258] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:23.682973 30049 ts_tablet_manager.cc:1905] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:23.684573 30049 ts_tablet_manager.cc:1918] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:38:23.684654 30049 log.cc:1196] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/0a81860cc3a54577a0ebaf6346736f49
I20260928 14:38:23.684929 30049 ts_tablet_manager.cc:1939] T 0a81860cc3a54577a0ebaf6346736f49 P d844e25249cd4df5a06f0884bf9850f5: Deleting consensus metadata
I20260928 14:38:23.685319 29457 catalog_manager.cc:4841] TS d844e25249cd4df5a06f0884bf9850f5 (127.22.13.66:36965): tablet 0a81860cc3a54577a0ebaf6346736f49 (table test-table [id=a5ec7f8dc0e74db4a4e2467f8ec81cab]) successfully deleted
W20260928 14:38:23.687532 30016 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:23.687339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:41419 (local address 127.0.0.1:52556)
0928 14:38:23.687408 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:23.687421 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:23.687479 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:41419: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":42}
W20260928 14:38:23.687692 29457 catalog_manager.cc:4584] TS 456ec803412d4816b65345ebf7a8d8a5 (127.22.13.65:41419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a81860cc3a54577a0ebaf6346736f49: Network error: Client connection negotiation failed: client connection to 127.22.13.65:41419: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260928 14:38:23.687965 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 29658
I20260928 14:38:23.688840 30047 ts_tablet_manager.cc:1905] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:23.690234 30047 ts_tablet_manager.cc:1918] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:38:23.690300 30047 log.cc:1196] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/0a81860cc3a54577a0ebaf6346736f49
I20260928 14:38:23.690531 30047 ts_tablet_manager.cc:1939] T 0a81860cc3a54577a0ebaf6346736f49 P ac34180e0f1c455da81fb0c4bc578d4b: Deleting consensus metadata
I20260928 14:38:23.690914 29458 catalog_manager.cc:4841] TS ac34180e0f1c455da81fb0c4bc578d4b (127.22.13.67:38887): tablet 0a81860cc3a54577a0ebaf6346736f49 (table test-table [id=a5ec7f8dc0e74db4a4e2467f8ec81cab]) successfully deleted
I20260928 14:38:23.695243 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 29794
I20260928 14:38:23.700307 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 29149
I20260928 14:38:23.705485 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 29337
2026-09-28T14:38:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6476 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:38:23 dist-test-slave-wwwb krb5kdc[30052](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:38:23 dist-test-slave-wwwb krb5kdc[30052](info): set up 2 sockets
Sep 28 14:38:23 dist-test-slave-wwwb krb5kdc[30052](info): commencing operation
krb5kdc: starting...
W20260928 14:38:24.722775 22581 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:38:24 dist-test-slave-wwwb krb5kdc[30052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:38:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:24Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.126.keytab.
Entry for principal oryx/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.126.keytab.
I20260928 14:38:24.842543 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:39087
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.126.keytab
--principal=oryx/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:38485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:39087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:24.924046 30065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:24.924288 30065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:24.924311 30065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:24.925442 30065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:24.925479 30065 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 14:38:24.925493 30065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:24.925506 30065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:24.925518 30065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:24.925529 30065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:24.927209 30065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:38485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:39087
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.126.keytab
--principal=oryx/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:39087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:24.927428 30065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:24.927623 30065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:24.932653 30070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:24.932660 30073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:24.932679 30065 server_base.cc:1034] running on GCE node
W20260928 14:38:24.932660 30071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:24.933162 30065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:24.933432 30065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:24.934572 30065 hybrid_clock.cc:648] HybridClock initialized: now 1790606304934550 us; error 41 us; skew 500 ppm
Sep 28 14:38:24 dist-test-slave-wwwb krb5kdc[30052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606304, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:24.937038 30065 init.cc:377] Logged in from keytab as oryx/127.22.13.126@KRBTEST.COM (short username oryx)
I20260928 14:38:24.937536 30065 webserver.cc:460] Webserver started at http://127.22.13.126:41119/ using document root <none> and password file <none>
I20260928 14:38:24.937750 30065 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:24.937793 30065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:24.937884 30065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:24.938995 30065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "e93b8d2ebe21440a9ec81550a23d398b"
format_stamp: "Formatted at 2026-09-28 14:38:24 on dist-test-slave-wwwb"
server_key: "f6f13c79aec53b704435f88b07415d21"
server_key_iv: "28c91941096db376ad2add537e41a605"
server_key_version: "encryptionkey@0"
I20260928 14:38:24.939294 30065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "e93b8d2ebe21440a9ec81550a23d398b"
format_stamp: "Formatted at 2026-09-28 14:38:24 on dist-test-slave-wwwb"
server_key: "f6f13c79aec53b704435f88b07415d21"
server_key_iv: "28c91941096db376ad2add537e41a605"
server_key_version: "encryptionkey@0"
I20260928 14:38:24.940595 30065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:38:24.941295 30080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:24.941494 30065 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:24.941581 30065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "e93b8d2ebe21440a9ec81550a23d398b"
format_stamp: "Formatted at 2026-09-28 14:38:24 on dist-test-slave-wwwb"
server_key: "f6f13c79aec53b704435f88b07415d21"
server_key_iv: "28c91941096db376ad2add537e41a605"
server_key_version: "encryptionkey@0"
I20260928 14:38:24.941648 30065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:24.951947 30065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:24.954170 30065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:24.954293 30065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:24.958918 30065 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:39087
I20260928 14:38:24.958999 30142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:39087 every 8 connection(s)
I20260928 14:38:24.959363 30065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:24.960127 30143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:24.962515 30143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b: Bootstrap starting.
I20260928 14:38:24.963158 30143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:24.963464 30143 log.cc:824] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:24.964180 30143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b: No bootstrap required, opened a new log
I20260928 14:38:24.966364 30143 raft_consensus.cc:348] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93b8d2ebe21440a9ec81550a23d398b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39087 } }
I20260928 14:38:24.966539 30143 raft_consensus.cc:374] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:24.966590 30143 raft_consensus.cc:729] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e93b8d2ebe21440a9ec81550a23d398b, State: Initialized, Role: FOLLOWER
I20260928 14:38:24.966763 30143 consensus_queue.cc:260] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93b8d2ebe21440a9ec81550a23d398b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39087 } }
I20260928 14:38:24.966848 30143 raft_consensus.cc:388] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:24.966894 30143 raft_consensus.cc:482] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:24.966943 30143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:24.967512 30143 raft_consensus.cc:504] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93b8d2ebe21440a9ec81550a23d398b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39087 } }
I20260928 14:38:24.967656 30143 leader_election.cc:302] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e93b8d2ebe21440a9ec81550a23d398b; no voters:
I20260928 14:38:24.967825 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 30065
I20260928 14:38:24.967869 30143 leader_election.cc:288] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:24.967898 30146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:24.967967 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:24.968056 30146 raft_consensus.cc:686] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [term 1 LEADER]: Becoming Leader. State: Replica: e93b8d2ebe21440a9ec81550a23d398b, State: Running, Role: LEADER
I20260928 14:38:24.968142 30146 consensus_queue.cc:237] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93b8d2ebe21440a9ec81550a23d398b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39087 } }
I20260928 14:38:24.968290 22581 external_mini_cluster.cc:1428] Setting key dcdb165384ef115a6e1fd2a12d6b770b
I20260928 14:38:24.968377 30143 sys_catalog.cc:564] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:24.968597 30147 sys_catalog.cc:455] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e93b8d2ebe21440a9ec81550a23d398b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93b8d2ebe21440a9ec81550a23d398b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39087 } } }
I20260928 14:38:24.968683 30147 sys_catalog.cc:458] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:24.968631 30148 sys_catalog.cc:455] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e93b8d2ebe21440a9ec81550a23d398b. Latest consensus state: current_term: 1 leader_uuid: "e93b8d2ebe21440a9ec81550a23d398b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e93b8d2ebe21440a9ec81550a23d398b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39087 } } }
I20260928 14:38:24.968722 30148 sys_catalog.cc:458] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:24.969041 30151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:24.969882 30151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:24.971593 30151 catalog_manager.cc:1347] Generated new cluster ID: fde52f27338a4191bd2d4952f3ea1a39
I20260928 14:38:24.971650 30151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:38:24 dist-test-slave-wwwb krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM
I20260928 14:38:24.980535 30156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:24.968898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43564 (local address 127.22.13.126:39087)
0928 14:38:24.969415 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:24.969426 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:24.969462 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:38:24.969618 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:24.969625 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:24.969685 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:24.969877 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:24.970519 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:24.971917 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:24.973042 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:24.973686 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:24.973800 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:24.976704 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:24.976710 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:24.976710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:24.976723 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:24.978750 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:24.979177 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:24.979182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:24.979182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:24.979308 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:24.979516 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:24.979516 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:24.979517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:24.979881 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:24.980052 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:24.980137 (+ 85us) server_negotiation.cc:299] Negotiation successful
0928 14:38:24.980291 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":69,"threads_started":1}
I20260928 14:38:24.991176 30151 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:24.991752 30151 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:24.996003 30151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b: Generated new TSK 0
I20260928 14:38:24.996229 30151 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.65.keytab.
Entry for principal oryx/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.65.keytab.
I20260928 14:38:25.024765 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:39087
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.65.keytab
--principal=oryx/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:38485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:25.105587 30169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:25.105841 30169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:25.105883 30169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:25.107129 30169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:25.107206 30169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:25.107242 30169 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:38:25.107282 30169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:38:25.108810 30169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:38485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.65.keytab
--principal=oryx/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.13.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:25.109122 30169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:25.109361 30169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:25.114331 30174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:25.114384 30175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:25.114389 30177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:25.114632 30169 server_base.cc:1034] running on GCE node
I20260928 14:38:25.114848 30169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:25.115104 30169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:25.116252 30169 hybrid_clock.cc:648] HybridClock initialized: now 1790606305116232 us; error 41 us; skew 500 ppm
Sep 28 14:38:25 dist-test-slave-wwwb krb5kdc[30052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606305, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:25.118988 30169 init.cc:377] Logged in from keytab as oryx/127.22.13.65@KRBTEST.COM (short username oryx)
I20260928 14:38:25.119457 30169 webserver.cc:460] Webserver started at http://127.22.13.65:41249/ using document root <none> and password file <none>
I20260928 14:38:25.119714 30169 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:25.119762 30169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:25.119932 30169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:25.120998 30169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "715fe73a1af64033b70315b19c0e074c"
format_stamp: "Formatted at 2026-09-28 14:38:25 on dist-test-slave-wwwb"
server_key: "9cb65e7877d3f60dc1a3bcb3edb8f008"
server_key_iv: "161ad9e258b080ab46ada85f917f4306"
server_key_version: "encryptionkey@0"
I20260928 14:38:25.121348 30169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "715fe73a1af64033b70315b19c0e074c"
format_stamp: "Formatted at 2026-09-28 14:38:25 on dist-test-slave-wwwb"
server_key: "9cb65e7877d3f60dc1a3bcb3edb8f008"
server_key_iv: "161ad9e258b080ab46ada85f917f4306"
server_key_version: "encryptionkey@0"
I20260928 14:38:25.122795 30169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:25.123553 30184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:25.123726 30169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:25.123806 30169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "715fe73a1af64033b70315b19c0e074c"
format_stamp: "Formatted at 2026-09-28 14:38:25 on dist-test-slave-wwwb"
server_key: "9cb65e7877d3f60dc1a3bcb3edb8f008"
server_key_iv: "161ad9e258b080ab46ada85f917f4306"
server_key_version: "encryptionkey@0"
I20260928 14:38:25.123860 30169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:25.135247 30169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:25.137195 30169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:25.137311 30169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:25.137771 30169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:25.137806 30169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:25.137863 30169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:25.137892 30169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:25.144840 30169 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:37765
I20260928 14:38:25.144903 30299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:37765 every 8 connection(s)
Sep 28 14:38:25 dist-test-slave-wwwb krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606305, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM
I20260928 14:38:25.145268 30169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:38:25.148521 30156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.138383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:38907 (local address 127.22.13.126:39087)
0928 14:38:25.138424 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.138427 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.138656 (+ 229us) server_negotiation.cc:407] Connection header received
0928 14:38:25.139228 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.139230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.139249 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.139329 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:25.140285 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.141156 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.142613 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.142963 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.143051 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.145783 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:25.145785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:25.145785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:25.145801 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:25.147328 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:25.147713 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:25.147714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:25.147714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:25.147765 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:25.147985 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:25.147985 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:25.147985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:25.148086 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:25.148141 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:25.148376 (+ 235us) server_negotiation.cc:299] Negotiation successful
0928 14:38:25.148455 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:38:25.148681 30193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.138394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39087 (local address 127.22.13.65:38907)
0928 14:38:25.138537 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:25.138570 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:25.139070 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:25.139417 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:25.139434 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:25.139824 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:25.140174 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.140182 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.141289 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.141290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.142496 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.142499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.143062 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.143063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.143204 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.143769 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:25.143782 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:25.145668 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:25.147402 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:25.147408 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:25.147410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:25.147642 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:25.147821 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:25.147822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:25.147822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:25.147933 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:25.148145 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:25.148149 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:25.148266 (+ 117us) client_negotiation.cc:769] Sending connection context
0928 14:38:25.148350 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 14:38:25.148473 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260928 14:38:25.149168 30300 heartbeater.cc:347] Connected to a master server at 127.22.13.126:39087
I20260928 14:38:25.149266 30300 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:25.149459 30300 heartbeater.cc:511] Master 127.22.13.126:39087 requested a full tablet report, sending...
I20260928 14:38:25.150058 30096 ts_manager.cc:194] Registered new tserver with Master: 715fe73a1af64033b70315b19c0e074c (127.22.13.65:37765)
I20260928 14:38:25.150115 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 30169
I20260928 14:38:25.150203 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:38:25.150395 22581 external_mini_cluster.cc:1428] Setting key b69c74525df9dc27eb899699c792da22
I20260928 14:38:25.150941 30096 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:38907
I20260928 14:38:25.156329 30156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.150309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:39195 (local address 127.22.13.126:39087)
0928 14:38:25.150348 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.150350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.150495 (+ 145us) server_negotiation.cc:407] Connection header received
0928 14:38:25.150586 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.150587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.150596 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.150651 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:25.151081 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.151695 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.152410 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.152624 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.152702 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.153712 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:25.153713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:25.153713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:25.153720 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:25.155156 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:25.155531 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:25.155533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:25.155533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:25.155588 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:25.155798 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:25.155799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:25.155799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:25.155917 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:25.155991 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:25.156150 (+ 159us) server_negotiation.cc:299] Negotiation successful
0928 14:38:25.156234 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:38:25.156333 30306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.150263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39087 (local address 127.22.13.65:39195)
0928 14:38:25.150418 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:25.150428 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:25.150507 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:25.150717 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:25.150718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:25.150916 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:25.150984 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.150986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.151831 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.151832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.152326 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.152327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.152711 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.152712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.152779 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.153274 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:25.153279 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:25.153590 (+ 311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:25.155261 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:25.155262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:25.155263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:25.155445 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:25.155662 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:25.155663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:25.155663 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:25.155722 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:25.155982 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:25.155983 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:25.156032 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 14:38:25.156118 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 14:38:25.156232 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for oryx/127.22.13.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.66.keytab.
Entry for principal oryx/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.66.keytab.
I20260928 14:38:25.176385 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:39087
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.66.keytab
--principal=oryx/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:38485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:25.258751 30309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:25.258962 30309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:25.258986 30309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:25.260064 30309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:25.260102 30309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:25.260121 30309 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:38:25.260138 30309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:38:25.261641 30309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:38485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.66.keytab
--principal=oryx/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.13.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:25.261835 30309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:25.262039 30309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:25.267040 30315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:25.267047 30317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:25.267084 30314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:25.267318 30309 server_base.cc:1034] running on GCE node
I20260928 14:38:25.267643 30309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:25.267912 30309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:25.269116 30309 hybrid_clock.cc:648] HybridClock initialized: now 1790606305269095 us; error 33 us; skew 500 ppm
Sep 28 14:38:25 dist-test-slave-wwwb krb5kdc[30052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606305, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:25.271745 30309 init.cc:377] Logged in from keytab as oryx/127.22.13.66@KRBTEST.COM (short username oryx)
I20260928 14:38:25.272241 30309 webserver.cc:460] Webserver started at http://127.22.13.66:36851/ using document root <none> and password file <none>
I20260928 14:38:25.272501 30309 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:25.272569 30309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:25.272745 30309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:25.273816 30309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "284ea2ca848a490babfc9c64248c8b7a"
format_stamp: "Formatted at 2026-09-28 14:38:25 on dist-test-slave-wwwb"
server_key: "dbba9c8150cce0be3d0a4965f9361413"
server_key_iv: "1c893cf549a906a7ba36ea0b4d1b0113"
server_key_version: "encryptionkey@0"
I20260928 14:38:25.274255 30309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "284ea2ca848a490babfc9c64248c8b7a"
format_stamp: "Formatted at 2026-09-28 14:38:25 on dist-test-slave-wwwb"
server_key: "dbba9c8150cce0be3d0a4965f9361413"
server_key_iv: "1c893cf549a906a7ba36ea0b4d1b0113"
server_key_version: "encryptionkey@0"
I20260928 14:38:25.275604 30309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:25.276317 30324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:25.276476 30309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:25.276582 30309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "284ea2ca848a490babfc9c64248c8b7a"
format_stamp: "Formatted at 2026-09-28 14:38:25 on dist-test-slave-wwwb"
server_key: "dbba9c8150cce0be3d0a4965f9361413"
server_key_iv: "1c893cf549a906a7ba36ea0b4d1b0113"
server_key_version: "encryptionkey@0"
I20260928 14:38:25.276662 30309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:25.302979 30309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:25.304904 30309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:25.305085 30309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:25.305689 30309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:25.305751 30309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:25.305797 30309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:25.305820 30309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:38:25 dist-test-slave-wwwb krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606305, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM
I20260928 14:38:25.313442 30309 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:37585
I20260928 14:38:25.313516 30439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:37585 every 8 connection(s)
I20260928 14:38:25.313853 30309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:38:25.316063 30156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.306148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:58285 (local address 127.22.13.126:39087)
0928 14:38:25.306219 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.306222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.306506 (+ 284us) server_negotiation.cc:407] Connection header received
0928 14:38:25.307342 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.307343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.307354 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.307395 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:25.308449 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.308956 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.310036 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.310241 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.310319 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.313341 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:25.313343 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:25.313343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:25.313352 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:25.314898 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:25.315281 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:25.315282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:25.315282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:25.315350 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:25.315585 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:25.315585 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:25.315586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:25.315700 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:25.315748 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:25.315943 (+ 195us) server_negotiation.cc:299] Negotiation successful
0928 14:38:25.316003 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:38:25.316256 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.306142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39087 (local address 127.22.13.66:58285)
0928 14:38:25.306343 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:25.306405 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:25.307199 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:25.307495 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:25.307501 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:25.307960 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:25.308326 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.308334 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.309061 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.309062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.309910 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.309913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.310341 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.310343 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.310469 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.311032 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:25.311041 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:25.313238 (+ 2197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:25.314978 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:25.314982 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:25.314984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:25.315206 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:25.315405 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:25.315406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:25.315406 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:25.315517 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:25.315740 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:25.315742 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:25.315855 (+ 113us) client_negotiation.cc:769] Sending connection context
0928 14:38:25.315930 (+ 75us) client_negotiation.cc:240] Negotiation successful
0928 14:38:25.316050 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":93,"threads_started":1}
I20260928 14:38:25.316728 30440 heartbeater.cc:347] Connected to a master server at 127.22.13.126:39087
I20260928 14:38:25.316830 30440 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:25.317010 30440 heartbeater.cc:511] Master 127.22.13.126:39087 requested a full tablet report, sending...
I20260928 14:38:25.317444 30097 ts_manager.cc:194] Registered new tserver with Master: 284ea2ca848a490babfc9c64248c8b7a (127.22.13.66:37585)
I20260928 14:38:25.317974 30097 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:58285
I20260928 14:38:25.322181 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 30309
I20260928 14:38:25.322326 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:38:25.322558 22581 external_mini_cluster.cc:1428] Setting key f190b6ab7ae6ca941720634fd31c3e39
I20260928 14:38:25.323189 30156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.317759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:54731 (local address 127.22.13.126:39087)
0928 14:38:25.317797 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.317798 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.317854 (+ 56us) server_negotiation.cc:407] Connection header received
0928 14:38:25.317913 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.317914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.317921 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.318028 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:25.318384 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.319001 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.319653 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.319892 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.319942 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.320799 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:25.320800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:25.320800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:25.320804 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:25.321885 (+ 1081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:25.322330 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:25.322332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:25.322332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:25.322396 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:25.322584 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:25.322584 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:25.322585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:25.322778 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:25.322859 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:25.323024 (+ 165us) server_negotiation.cc:299] Negotiation successful
0928 14:38:25.323090 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:38:25.323237 30446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.317667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39087 (local address 127.22.13.66:54731)
0928 14:38:25.317793 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:25.317804 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:25.317861 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:25.318071 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:25.318072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:25.318264 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:25.318314 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.318316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.319088 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.319089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.319578 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.319579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.319955 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.319956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.320001 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.320432 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:25.320438 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:25.320723 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:25.322003 (+ 1280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:25.322004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:25.322005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:25.322224 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:25.322458 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:25.322458 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:25.322459 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:25.322528 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:25.322853 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:25.322854 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:25.322923 (+ 69us) client_negotiation.cc:769] Sending connection context
0928 14:38:25.323003 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 14:38:25.323118 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for oryx/127.22.13.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.67.keytab.
Entry for principal oryx/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.67.keytab.
I20260928 14:38:25.348343 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:39087
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.67.keytab
--principal=oryx/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:38485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:25.429476 30449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:25.429705 30449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:25.429733 30449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:25.430859 30449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:25.430903 30449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:25.430923 30449 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:38:25.430943 30449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:38:25.432451 30449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:38485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.67.keytab
--principal=oryx/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.13.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:25.432672 30449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:25.432873 30449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:25.438257 30455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:25.438263 30454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:25.438265 30457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:25.438535 30449 server_base.cc:1034] running on GCE node
I20260928 14:38:25.438771 30449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:25.438973 30449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:25.440115 30449 hybrid_clock.cc:648] HybridClock initialized: now 1790606305440096 us; error 34 us; skew 500 ppm
Sep 28 14:38:25 dist-test-slave-wwwb krb5kdc[30052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606305, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:25.442941 30449 init.cc:377] Logged in from keytab as oryx/127.22.13.67@KRBTEST.COM (short username oryx)
I20260928 14:38:25.443475 30449 webserver.cc:460] Webserver started at http://127.22.13.67:35939/ using document root <none> and password file <none>
I20260928 14:38:25.443727 30449 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:25.443774 30449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:25.443954 30449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:25.445154 30449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "5563db9576244444bd9ba2c9c492855b"
format_stamp: "Formatted at 2026-09-28 14:38:25 on dist-test-slave-wwwb"
server_key: "f9fe6d73529c166f8bca11e597537940"
server_key_iv: "111e8dd4049a724ad2068f4ae2df18b9"
server_key_version: "encryptionkey@0"
I20260928 14:38:25.445533 30449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "5563db9576244444bd9ba2c9c492855b"
format_stamp: "Formatted at 2026-09-28 14:38:25 on dist-test-slave-wwwb"
server_key: "f9fe6d73529c166f8bca11e597537940"
server_key_iv: "111e8dd4049a724ad2068f4ae2df18b9"
server_key_version: "encryptionkey@0"
I20260928 14:38:25.447217 30449 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:25.447988 30464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:25.448168 30449 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:25.448278 30449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "5563db9576244444bd9ba2c9c492855b"
format_stamp: "Formatted at 2026-09-28 14:38:25 on dist-test-slave-wwwb"
server_key: "f9fe6d73529c166f8bca11e597537940"
server_key_iv: "111e8dd4049a724ad2068f4ae2df18b9"
server_key_version: "encryptionkey@0"
I20260928 14:38:25.448355 30449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:25.471907 30449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:25.474442 30449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:25.474620 30449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:25.475147 30449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:25.475212 30449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:25.475260 30449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:25.475286 30449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:38:25 dist-test-slave-wwwb krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606305, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM
I20260928 14:38:25.482321 30449 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:35827
I20260928 14:38:25.482746 30449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:38:25.483253 30579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:35827 every 8 connection(s)
I20260928 14:38:25.483851 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 30449
I20260928 14:38:25.483958 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:38:25.484141 22581 external_mini_cluster.cc:1428] Setting key d3d4475978b63c45a1e03bcfbd79536a
I20260928 14:38:25.485810 30156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.475707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:48185 (local address 127.22.13.126:39087)
0928 14:38:25.475782 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.475785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.475957 (+ 172us) server_negotiation.cc:407] Connection header received
0928 14:38:25.476450 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.476451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.476468 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.476546 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:25.477460 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.478356 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.479516 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.479854 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.479952 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.482847 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:25.482848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:25.482848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:25.482855 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:25.484520 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:25.485020 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:25.485021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:25.485021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:25.485087 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:25.485326 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:25.485327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:25.485327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:25.485429 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:25.485484 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:25.485686 (+ 202us) server_negotiation.cc:299] Negotiation successful
0928 14:38:25.485749 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:38:25.486100 30473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.475675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39087 (local address 127.22.13.67:48185)
0928 14:38:25.475826 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:25.475867 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:25.476354 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:25.476650 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:25.476653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:25.477031 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:25.477342 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.477349 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.478463 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.478464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.479430 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.479438 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.479967 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.479968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.480115 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.481100 (+ 985us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:25.481108 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:25.482718 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:25.484607 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:25.484613 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:25.484616 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:25.484912 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:25.485153 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:25.485154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:25.485154 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:25.485271 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:25.485479 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:25.485483 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:25.485597 (+ 114us) client_negotiation.cc:769] Sending connection context
0928 14:38:25.485675 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:38:25.485800 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260928 14:38:25.486570 30580 heartbeater.cc:347] Connected to a master server at 127.22.13.126:39087
I20260928 14:38:25.486685 30580 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:25.486876 30580 heartbeater.cc:511] Master 127.22.13.126:39087 requested a full tablet report, sending...
I20260928 14:38:25.487344 30096 ts_manager.cc:194] Registered new tserver with Master: 5563db9576244444bd9ba2c9c492855b (127.22.13.67:35827)
I20260928 14:38:25.487859 30096 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:48185
I20260928 14:38:25.493208 30586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.487667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39087 (local address 127.22.13.67:32949)
0928 14:38:25.487822 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:25.487831 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:25.487903 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:25.488042 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:25.488043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:25.488233 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:25.488295 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.488297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.488949 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.488950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.489468 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.489469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.489847 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.489848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.489914 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.490427 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:25.490431 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:25.490727 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:25.492102 (+ 1375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:25.492103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:25.492104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:25.492278 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:25.492496 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:25.492497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:25.492497 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:25.492557 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:25.492828 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:25.492829 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:25.492876 (+ 47us) client_negotiation.cc:769] Sending connection context
0928 14:38:25.492958 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 14:38:25.493099 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
I20260928 14:38:25.493196 30156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.487717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:32949 (local address 127.22.13.126:39087)
0928 14:38:25.487757 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.487759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.487888 (+ 129us) server_negotiation.cc:407] Connection header received
0928 14:38:25.487971 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.487971 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.487981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.488037 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:25.488384 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.488857 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.489554 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.489762 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.489845 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.490836 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:25.490838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:25.490838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:25.490845 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:25.492004 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:25.492363 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:25.492364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:25.492365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:25.492420 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:25.492637 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:25.492638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:25.492638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:25.492752 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:25.492822 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:25.492992 (+ 170us) server_negotiation.cc:299] Negotiation successful
0928 14:38:25.493097 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:38:25.495937 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 14:38:25 dist-test-slave-wwwb krb5kdc[30052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606305, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 14:38:25 dist-test-slave-wwwb krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:25 dist-test-slave-wwwb krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
W20260928 14:38:25.532963 30156 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:25.516045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43580 (local address 127.22.13.126:39087)
0928 14:38:25.516120 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.516124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.516132 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:38:25.516166 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.516167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.516184 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.516254 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:25.517013 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.517530 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.518444 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.518648 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.518744 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.532840 (+ 14096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43580: BlockingRecv error: recv got EOF from 127.0.0.1:43580 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 28 14:38:25 dist-test-slave-wwwb krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606305, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM
I20260928 14:38:25.540738 30156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.533748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43586 (local address 127.22.13.126:39087)
0928 14:38:25.533819 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.533822 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.533879 (+ 57us) server_negotiation.cc:407] Connection header received
0928 14:38:25.533972 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.533973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.533986 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.534075 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:25.534575 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.535073 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.535793 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.535987 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.536094 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.538365 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:25.538366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:25.538367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:25.538375 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:25.539558 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:25.539927 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:25.539929 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:25.539929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:25.539992 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:25.540196 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:25.540197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:25.540197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:25.540318 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:25.540427 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:25.540522 (+ 95us) server_negotiation.cc:299] Negotiation successful
0928 14:38:25.540620 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 14:38:25.542678 30096 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:38:25.544351 30096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:38:25.551388 30603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.546750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:37585 (local address 127.0.0.1:46714)
0928 14:38:25.546908 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:25.546969 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:25.547043 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:25.547664 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:25.547667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:25.547669 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:25.547746 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.547749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.549149 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.549150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.550283 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.550285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.550998 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.550999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.551115 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.551131 (+ 16us) client_negotiation.cc:769] Sending connection context
0928 14:38:25.551235 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 14:38:25.551297 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260928 14:38:25.551436 30607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.547249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46714 (local address 127.22.13.66:37585)
0928 14:38:25.547472 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.547478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.547489 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:38:25.547528 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.547530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.547545 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.547651 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:25.547833 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.549033 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.550371 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.550884 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.550995 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.551082 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:25.551292 (+ 210us) server_negotiation.cc:299] Negotiation successful
0928 14:38:25.551332 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":96,"threads_started":1}
I20260928 14:38:25.551628 30602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.546756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:37765 (local address 127.0.0.1:38958)
0928 14:38:25.547244 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:25.547255 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:25.547309 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:25.547853 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:25.547854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:25.547855 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:25.547881 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.547882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.548815 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.548816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.550259 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.550263 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.551315 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.551316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.551379 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.551384 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:38:25.551471 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 14:38:25.551525 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":27,"thread_start_us":101,"threads_started":1}
I20260928 14:38:25.551772 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.546750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:35827 (local address 127.0.0.1:36018)
0928 14:38:25.547154 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:25.547164 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:25.547201 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:25.547738 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:25.547739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:25.547740 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:25.547780 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.547781 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.549141 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.549142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.550274 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.550277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.551535 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.551537 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.551608 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.551613 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:38:25.551664 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 14:38:25.551705 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":36,"thread_start_us":65,"threads_started":1}
I20260928 14:38:25.552084 30605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.547020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36018 (local address 127.22.13.67:35827)
0928 14:38:25.547230 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.547236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.547247 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:38:25.547290 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.547293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.547310 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.547388 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:25.547859 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.549034 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.550386 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.551146 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.551210 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.551309 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:25.551888 (+ 579us) server_negotiation.cc:299] Negotiation successful
0928 14:38:25.551935 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":112,"threads_started":1}
I20260928 14:38:25.552374 30374 tablet_service.cc:1467] Processing CreateTablet for tablet 961ade997eb8461aab81e607c4bb223d (DEFAULT_TABLE table=test-table [id=3ffcd134a21243388a4787e7bab1bedc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:25.552707 30374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 961ade997eb8461aab81e607c4bb223d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:25.552702 30514 tablet_service.cc:1467] Processing CreateTablet for tablet 961ade997eb8461aab81e607c4bb223d (DEFAULT_TABLE table=test-table [id=3ffcd134a21243388a4787e7bab1bedc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:25.552970 30514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 961ade997eb8461aab81e607c4bb223d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:25.554464 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.547061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38958 (local address 127.22.13.65:37765)
0928 14:38:25.547534 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.547540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.547549 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:38:25.547587 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.547589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.547604 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.547676 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:25.547963 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.548731 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.550361 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.551114 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.551190 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.551274 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:25.554306 (+ 3032us) server_negotiation.cc:299] Negotiation successful
0928 14:38:25.554362 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":96,"threads_started":1}
I20260928 14:38:25.555168 30234 tablet_service.cc:1467] Processing CreateTablet for tablet 961ade997eb8461aab81e607c4bb223d (DEFAULT_TABLE table=test-table [id=3ffcd134a21243388a4787e7bab1bedc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:25.555595 30234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 961ade997eb8461aab81e607c4bb223d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:25.556171 30608 tablet_bootstrap.cc:492] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a: Bootstrap starting.
I20260928 14:38:25.556737 30608 tablet_bootstrap.cc:654] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:25.557058 30608 log.cc:824] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:25.557776 30608 tablet_bootstrap.cc:492] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a: No bootstrap required, opened a new log
I20260928 14:38:25.557864 30608 ts_tablet_manager.cc:1397] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:38:25.558342 30609 tablet_bootstrap.cc:492] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b: Bootstrap starting.
I20260928 14:38:25.559063 30609 tablet_bootstrap.cc:654] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:25.559150 30611 tablet_bootstrap.cc:492] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c: Bootstrap starting.
I20260928 14:38:25.559520 30609 log.cc:824] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:25.559689 30611 tablet_bootstrap.cc:654] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:25.560024 30611 log.cc:824] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:25.560269 30608 raft_consensus.cc:348] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.560459 30608 raft_consensus.cc:374] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:25.560508 30608 raft_consensus.cc:729] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284ea2ca848a490babfc9c64248c8b7a, State: Initialized, Role: FOLLOWER
I20260928 14:38:25.560652 30609 tablet_bootstrap.cc:492] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b: No bootstrap required, opened a new log
I20260928 14:38:25.560637 30608 consensus_queue.cc:260] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.560719 30609 ts_tablet_manager.cc:1397] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:38:25.560758 30611 tablet_bootstrap.cc:492] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c: No bootstrap required, opened a new log
I20260928 14:38:25.560813 30611 ts_tablet_manager.cc:1397] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:38:25.560928 30608 ts_tablet_manager.cc:1428] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 14:38:25.561074 30440 heartbeater.cc:503] Master 127.22.13.126:39087 was elected leader, sending a full tablet report...
I20260928 14:38:25.562763 30609 raft_consensus.cc:348] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.562767 30611 raft_consensus.cc:348] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.562889 30609 raft_consensus.cc:374] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:25.562932 30611 raft_consensus.cc:374] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:25.562953 30609 raft_consensus.cc:729] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5563db9576244444bd9ba2c9c492855b, State: Initialized, Role: FOLLOWER
I20260928 14:38:25.562983 30611 raft_consensus.cc:729] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 715fe73a1af64033b70315b19c0e074c, State: Initialized, Role: FOLLOWER
I20260928 14:38:25.563078 30609 consensus_queue.cc:260] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.563102 30611 consensus_queue.cc:260] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.563360 30609 ts_tablet_manager.cc:1428] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:38:25.563361 30611 ts_tablet_manager.cc:1428] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:38:25.563373 30300 heartbeater.cc:503] Master 127.22.13.126:39087 was elected leader, sending a full tablet report...
I20260928 14:38:25.563373 30580 heartbeater.cc:503] Master 127.22.13.126:39087 was elected leader, sending a full tablet report...
W20260928 14:38:25.564668 30441 tablet.cc:2369] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:38:25.646252 30301 tablet.cc:2369] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:38:25.715765 30614 raft_consensus.cc:482] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:38:25.716002 30614 raft_consensus.cc:504] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.716426 30614 leader_election.cc:288] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 715fe73a1af64033b70315b19c0e074c (127.22.13.65:37765), 5563db9576244444bd9ba2c9c492855b (127.22.13.67:35827)
I20260928 14:38:25.719517 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.716690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:36763 (local address 127.22.13.65:37765)
0928 14:38:25.716735 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.716737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.716770 (+ 33us) server_negotiation.cc:407] Connection header received
0928 14:38:25.716852 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.716854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.716867 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.716937 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:25.717172 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.717691 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.718694 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.719121 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.719178 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.719185 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:25.719385 (+ 200us) server_negotiation.cc:299] Negotiation successful
0928 14:38:25.719434 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:38:25.719558 30605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.716771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:34879 (local address 127.22.13.67:35827)
0928 14:38:25.716812 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.716814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.716981 (+ 167us) server_negotiation.cc:407] Connection header received
0928 14:38:25.717021 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.717022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.717033 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.717093 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:25.717322 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.717793 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.718783 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.719190 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.719263 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.719270 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:25.719443 (+ 173us) server_negotiation.cc:299] Negotiation successful
0928 14:38:25.719490 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:38:25.719559 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.716649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:37765 (local address 127.22.13.66:36763)
0928 14:38:25.716714 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:25.716729 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:25.716787 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:25.716971 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:25.716973 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:25.716975 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:25.717066 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.717069 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.717773 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.717774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.718595 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.718598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.719208 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.719209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.719301 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.719305 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:38:25.719360 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 14:38:25.719416 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260928 14:38:25.719571 30617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.716650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:35827 (local address 127.22.13.66:34879)
0928 14:38:25.716887 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:25.716897 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:25.716955 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:25.717107 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:25.717108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:25.717109 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:25.717210 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.717212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.717874 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.717875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.718690 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.718692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.719280 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.719281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.719337 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.719340 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:38:25.719417 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:38:25.719455 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":53,"thread_start_us":117,"threads_started":1}
I20260928 14:38:25.719995 30534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "961ade997eb8461aab81e607c4bb223d" candidate_uuid: "284ea2ca848a490babfc9c64248c8b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5563db9576244444bd9ba2c9c492855b" is_pre_election: true
I20260928 14:38:25.719986 30254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "961ade997eb8461aab81e607c4bb223d" candidate_uuid: "284ea2ca848a490babfc9c64248c8b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "715fe73a1af64033b70315b19c0e074c" is_pre_election: true
I20260928 14:38:25.720162 30534 raft_consensus.cc:2454] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 284ea2ca848a490babfc9c64248c8b7a in term 0.
I20260928 14:38:25.720208 30254 raft_consensus.cc:2454] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 284ea2ca848a490babfc9c64248c8b7a in term 0.
I20260928 14:38:25.720647 30327 leader_election.cc:302] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 284ea2ca848a490babfc9c64248c8b7a, 5563db9576244444bd9ba2c9c492855b; no voters:
I20260928 14:38:25.720839 30614 raft_consensus.cc:2781] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:38:25.720896 30614 raft_consensus.cc:482] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:38:25.720930 30614 raft_consensus.cc:3037] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:25.721668 30614 raft_consensus.cc:504] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.721830 30614 leader_election.cc:288] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [CANDIDATE]: Term 1 election: Requested vote from peers 715fe73a1af64033b70315b19c0e074c (127.22.13.65:37765), 5563db9576244444bd9ba2c9c492855b (127.22.13.67:35827)
I20260928 14:38:25.722036 30534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "961ade997eb8461aab81e607c4bb223d" candidate_uuid: "284ea2ca848a490babfc9c64248c8b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5563db9576244444bd9ba2c9c492855b"
I20260928 14:38:25.722038 30254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "961ade997eb8461aab81e607c4bb223d" candidate_uuid: "284ea2ca848a490babfc9c64248c8b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "715fe73a1af64033b70315b19c0e074c"
I20260928 14:38:25.722118 30534 raft_consensus.cc:3037] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:25.722151 30254 raft_consensus.cc:3037] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:25.722808 30534 raft_consensus.cc:2454] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 284ea2ca848a490babfc9c64248c8b7a in term 1.
I20260928 14:38:25.722901 30254 raft_consensus.cc:2454] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 284ea2ca848a490babfc9c64248c8b7a in term 1.
I20260928 14:38:25.722975 30327 leader_election.cc:302] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 284ea2ca848a490babfc9c64248c8b7a, 5563db9576244444bd9ba2c9c492855b; no voters:
I20260928 14:38:25.723060 30614 raft_consensus.cc:2781] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:25.723210 30614 raft_consensus.cc:686] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [term 1 LEADER]: Becoming Leader. State: Replica: 284ea2ca848a490babfc9c64248c8b7a, State: Running, Role: LEADER
I20260928 14:38:25.723413 30614 consensus_queue.cc:237] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.724272 30095 catalog_manager.cc:5495] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a reported cstate change: term changed from 0 to 1, leader changed from <none> to 284ea2ca848a490babfc9c64248c8b7a (127.22.13.66). New cstate: current_term: 1 leader_uuid: "284ea2ca848a490babfc9c64248c8b7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } health_report { overall_health: UNKNOWN } } }
W20260928 14:38:25.733337 30581 tablet.cc:2369] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:38:25.734546 30607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.730966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46728 (local address 127.22.13.66:37585)
0928 14:38:25.731023 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.731025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.731032 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:38:25.731076 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.731077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.731095 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.731174 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:38:25.731335 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.732282 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.733185 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.733665 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.733739 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.733992 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:38:25.734158 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:38:25.734248 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:25.734421 (+ 173us) server_negotiation.cc:299] Negotiation successful
0928 14:38:25.734447 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:38:25.737056 30534 raft_consensus.cc:1261] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Refusing update from remote peer 284ea2ca848a490babfc9c64248c8b7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:25.737067 30254 raft_consensus.cc:1261] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c [term 1 FOLLOWER]: Refusing update from remote peer 284ea2ca848a490babfc9c64248c8b7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:25.737591 30614 consensus_queue.cc:1035] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:25.737782 30618 consensus_queue.cc:1035] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:25.740787 30622 mvcc.cc:205] Tried to move back new op lower bound from 7334323428296089600 to 7334323428243689472. Current Snapshot: MvccSnapshot[applied={T|T < 7334323428296089600}]
I20260928 14:38:25.741210 30623 mvcc.cc:205] Tried to move back new op lower bound from 7334323428296089600 to 7334323428243689472. Current Snapshot: MvccSnapshot[applied={T|T < 7334323428296089600}]
I20260928 14:38:25.742479 30096 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:25.742570 30096 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:25.743426 30096 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b: Sending DeleteTablet for 3 replicas of tablet 961ade997eb8461aab81e607c4bb223d
I20260928 14:38:25.743793 30374 tablet_service.cc:1514] Processing DeleteTablet for tablet 961ade997eb8461aab81e607c4bb223d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:25 UTC) from {username='oryx', principal='oryx/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:46714
I20260928 14:38:25.743784 30514 tablet_service.cc:1514] Processing DeleteTablet for tablet 961ade997eb8461aab81e607c4bb223d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:25 UTC) from {username='oryx', principal='oryx/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:36018
I20260928 14:38:25.743794 30234 tablet_service.cc:1514] Processing DeleteTablet for tablet 961ade997eb8461aab81e607c4bb223d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:25 UTC) from {username='oryx', principal='oryx/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:38958
I20260928 14:38:25.744285 30096 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:38:25.744470 30096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:38:25.744393 30633 tablet_replica.cc:330] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b: stopping tablet replica
I20260928 14:38:25.744704 30633 raft_consensus.cc:2229] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:25.744885 30633 raft_consensus.cc:2258] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:25.745082 30634 tablet_replica.cc:330] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a: stopping tablet replica
I20260928 14:38:25.745200 30634 raft_consensus.cc:2229] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:38:25.745352 30634 raft_consensus.cc:2258] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:25.745929 30514 tablet_service.cc:1467] Processing CreateTablet for tablet 47d9c245681141d797bb926fd0d6b3ab (DEFAULT_TABLE table=test-table [id=4f0b7fadec234ed6833a3ffbd68e3000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:25.746121 30514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47d9c245681141d797bb926fd0d6b3ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:25.746110 30632 tablet_replica.cc:330] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c: stopping tablet replica
I20260928 14:38:25.746218 30632 raft_consensus.cc:2229] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:25.746315 30632 raft_consensus.cc:2258] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:25.747062 30634 ts_tablet_manager.cc:1905] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:25.747682 30609 tablet_bootstrap.cc:492] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b: Bootstrap starting.
I20260928 14:38:25.747932 30374 tablet_service.cc:1467] Processing CreateTablet for tablet 47d9c245681141d797bb926fd0d6b3ab (DEFAULT_TABLE table=test-table [id=4f0b7fadec234ed6833a3ffbd68e3000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:25.748289 30234 tablet_service.cc:1467] Processing CreateTablet for tablet 47d9c245681141d797bb926fd0d6b3ab (DEFAULT_TABLE table=test-table [id=4f0b7fadec234ed6833a3ffbd68e3000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:25.748359 30609 tablet_bootstrap.cc:654] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:25.748418 30234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47d9c245681141d797bb926fd0d6b3ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:25.749267 30609 tablet_bootstrap.cc:492] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b: No bootstrap required, opened a new log
I20260928 14:38:25.749347 30609 ts_tablet_manager.cc:1397] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:38:25.749552 30609 raft_consensus.cc:348] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.749691 30609 raft_consensus.cc:374] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:25.749722 30609 raft_consensus.cc:729] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5563db9576244444bd9ba2c9c492855b, State: Initialized, Role: FOLLOWER
I20260928 14:38:25.749804 30609 consensus_queue.cc:260] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.749922 30611 tablet_bootstrap.cc:492] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c: Bootstrap starting.
I20260928 14:38:25.749981 30609 ts_tablet_manager.cc:1428] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:25.750334 30374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47d9c245681141d797bb926fd0d6b3ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:25.750347 30611 tablet_bootstrap.cc:654] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:25.751087 30611 tablet_bootstrap.cc:492] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c: No bootstrap required, opened a new log
I20260928 14:38:25.751140 30611 ts_tablet_manager.cc:1397] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:25.751230 30634 ts_tablet_manager.cc:1918] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:38:25.751294 30634 log.cc:1196] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/961ade997eb8461aab81e607c4bb223d
I20260928 14:38:25.751353 30611 raft_consensus.cc:348] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.751421 30611 raft_consensus.cc:374] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:25.751442 30611 raft_consensus.cc:729] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 715fe73a1af64033b70315b19c0e074c, State: Initialized, Role: FOLLOWER
I20260928 14:38:25.751483 30611 consensus_queue.cc:260] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.751544 30634 ts_tablet_manager.cc:1939] T 961ade997eb8461aab81e607c4bb223d P 284ea2ca848a490babfc9c64248c8b7a: Deleting consensus metadata
I20260928 14:38:25.751636 30611 ts_tablet_manager.cc:1428] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:25.751803 30608 tablet_bootstrap.cc:492] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a: Bootstrap starting.
I20260928 14:38:25.752267 30608 tablet_bootstrap.cc:654] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:25.752280 30084 catalog_manager.cc:4841] TS 284ea2ca848a490babfc9c64248c8b7a (127.22.13.66:37585): tablet 961ade997eb8461aab81e607c4bb223d (table test-table [id=3ffcd134a21243388a4787e7bab1bedc]) successfully deleted
I20260928 14:38:25.752645 30633 ts_tablet_manager.cc:1905] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:25.752869 30632 ts_tablet_manager.cc:1905] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:25.753574 30608 tablet_bootstrap.cc:492] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a: No bootstrap required, opened a new log
I20260928 14:38:25.753641 30608 ts_tablet_manager.cc:1397] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 14:38:25.753790 30608 raft_consensus.cc:348] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.753875 30608 raft_consensus.cc:374] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:25.753909 30608 raft_consensus.cc:729] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284ea2ca848a490babfc9c64248c8b7a, State: Initialized, Role: FOLLOWER
I20260928 14:38:25.753976 30608 consensus_queue.cc:260] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.754083 30608 ts_tablet_manager.cc:1428] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:25.754593 30633 ts_tablet_manager.cc:1918] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:38:25.754678 30633 log.cc:1196] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/961ade997eb8461aab81e607c4bb223d
I20260928 14:38:25.754933 30632 ts_tablet_manager.cc:1918] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:38:25.754973 30633 ts_tablet_manager.cc:1939] T 961ade997eb8461aab81e607c4bb223d P 5563db9576244444bd9ba2c9c492855b: Deleting consensus metadata
I20260928 14:38:25.755014 30632 log.cc:1196] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/wal/wals/961ade997eb8461aab81e607c4bb223d
I20260928 14:38:25.755280 30632 ts_tablet_manager.cc:1939] T 961ade997eb8461aab81e607c4bb223d P 715fe73a1af64033b70315b19c0e074c: Deleting consensus metadata
I20260928 14:38:25.755438 30083 catalog_manager.cc:4841] TS 5563db9576244444bd9ba2c9c492855b (127.22.13.67:35827): tablet 961ade997eb8461aab81e607c4bb223d (table test-table [id=3ffcd134a21243388a4787e7bab1bedc]) successfully deleted
I20260928 14:38:25.755750 30081 catalog_manager.cc:4841] TS 715fe73a1af64033b70315b19c0e074c (127.22.13.65:37765): tablet 961ade997eb8461aab81e607c4bb223d (table test-table [id=3ffcd134a21243388a4787e7bab1bedc]) successfully deleted
I20260928 14:38:25.765836 30616 raft_consensus.cc:482] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:38:25.765993 30616 raft_consensus.cc:504] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.766405 30616 leader_election.cc:288] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 284ea2ca848a490babfc9c64248c8b7a (127.22.13.66:37585), 5563db9576244444bd9ba2c9c492855b (127.22.13.67:35827)
I20260928 14:38:25.770798 30605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.766812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:59055 (local address 127.22.13.67:35827)
0928 14:38:25.766855 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.766857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.767061 (+ 204us) server_negotiation.cc:407] Connection header received
0928 14:38:25.767088 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.767089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.767098 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.767155 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:25.767350 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.768044 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.769710 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.770344 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.770422 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.770433 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:25.770674 (+ 241us) server_negotiation.cc:299] Negotiation successful
0928 14:38:25.770720 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:38:25.770861 30607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.766818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:50131 (local address 127.22.13.66:37585)
0928 14:38:25.766902 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.766904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.766913 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:38:25.766945 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.766946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.766957 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.767019 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:25.767231 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.768020 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.769754 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.770377 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.770453 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.770463 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:25.770733 (+ 270us) server_negotiation.cc:299] Negotiation successful
0928 14:38:25.770777 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 14:38:25.771049 30638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.766558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:35827 (local address 127.22.13.65:59055)
0928 14:38:25.766927 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:25.766935 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:25.766975 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:25.767161 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:25.767162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:25.767164 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:25.767234 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.767236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.768139 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.768140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.769607 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.769610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.770472 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.770473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.770584 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.770589 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:38:25.770904 (+ 315us) client_negotiation.cc:240] Negotiation successful
0928 14:38:25.770954 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":93,"thread_start_us":142,"threads_started":1}
I20260928 14:38:25.771287 30639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.766558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:37585 (local address 127.22.13.65:50131)
0928 14:38:25.766768 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:25.766781 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:25.766844 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:25.767042 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:25.767043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:25.767046 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:25.767142 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.767144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.768138 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.768139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.769606 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.769610 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.770538 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.770540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.770634 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.770639 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:38:25.771027 (+ 388us) client_negotiation.cc:240] Negotiation successful
0928 14:38:25.771082 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":59,"thread_start_us":128,"threads_started":1}
I20260928 14:38:25.771389 30534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47d9c245681141d797bb926fd0d6b3ab" candidate_uuid: "715fe73a1af64033b70315b19c0e074c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5563db9576244444bd9ba2c9c492855b" is_pre_election: true
I20260928 14:38:25.771486 30534 raft_consensus.cc:2454] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 715fe73a1af64033b70315b19c0e074c in term 0.
I20260928 14:38:25.771518 30394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47d9c245681141d797bb926fd0d6b3ab" candidate_uuid: "715fe73a1af64033b70315b19c0e074c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "284ea2ca848a490babfc9c64248c8b7a" is_pre_election: true
I20260928 14:38:25.771675 30394 raft_consensus.cc:2454] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 715fe73a1af64033b70315b19c0e074c in term 0.
I20260928 14:38:25.771725 30187 leader_election.cc:302] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5563db9576244444bd9ba2c9c492855b, 715fe73a1af64033b70315b19c0e074c; no voters:
I20260928 14:38:25.771941 30616 raft_consensus.cc:2781] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:38:25.771991 30616 raft_consensus.cc:482] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:38:25.772019 30616 raft_consensus.cc:3037] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:25.772989 30616 raft_consensus.cc:504] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.773172 30616 leader_election.cc:288] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [CANDIDATE]: Term 1 election: Requested vote from peers 284ea2ca848a490babfc9c64248c8b7a (127.22.13.66:37585), 5563db9576244444bd9ba2c9c492855b (127.22.13.67:35827)
I20260928 14:38:25.773437 30394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47d9c245681141d797bb926fd0d6b3ab" candidate_uuid: "715fe73a1af64033b70315b19c0e074c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "284ea2ca848a490babfc9c64248c8b7a"
I20260928 14:38:25.773478 30534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47d9c245681141d797bb926fd0d6b3ab" candidate_uuid: "715fe73a1af64033b70315b19c0e074c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5563db9576244444bd9ba2c9c492855b"
I20260928 14:38:25.773527 30394 raft_consensus.cc:3037] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:25.773561 30534 raft_consensus.cc:3037] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:25.774413 30534 raft_consensus.cc:2454] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 715fe73a1af64033b70315b19c0e074c in term 1.
I20260928 14:38:25.774533 30394 raft_consensus.cc:2454] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 715fe73a1af64033b70315b19c0e074c in term 1.
I20260928 14:38:25.774597 30187 leader_election.cc:302] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5563db9576244444bd9ba2c9c492855b, 715fe73a1af64033b70315b19c0e074c; no voters:
I20260928 14:38:25.774693 30616 raft_consensus.cc:2781] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:25.774833 30616 raft_consensus.cc:686] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [term 1 LEADER]: Becoming Leader. State: Replica: 715fe73a1af64033b70315b19c0e074c, State: Running, Role: LEADER
I20260928 14:38:25.774909 30616 consensus_queue.cc:237] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.775633 30095 catalog_manager.cc:5495] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c reported cstate change: term changed from 0 to 1, leader changed from <none> to 715fe73a1af64033b70315b19c0e074c (127.22.13.65). New cstate: current_term: 1 leader_uuid: "715fe73a1af64033b70315b19c0e074c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } health_report { overall_health: UNKNOWN } } }
Sep 28 14:38:25 dist-test-slave-wwwb krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606304, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.126@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM
I20260928 14:38:25.806869 30156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.798837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43594 (local address 127.22.13.126:39087)
0928 14:38:25.798896 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:25.798899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:25.799011 (+ 112us) server_negotiation.cc:407] Connection header received
0928 14:38:25.799077 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:25.799078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:25.799090 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:25.799157 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:25.799621 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.800403 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.801093 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.801401 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.801474 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.804146 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:25.804148 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:25.804148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:25.804154 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:25.805591 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:25.806108 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:25.806109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:25.806109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:25.806162 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:25.806365 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:25.806365 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:25.806365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:25.806472 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:25.806520 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:25.806729 (+ 209us) server_negotiation.cc:299] Negotiation successful
0928 14:38:25.806802 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 14:38:25.806984 30647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:25.798751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39087 (local address 127.0.0.1:43594)
0928 14:38:25.798920 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:25.798931 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:25.799011 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:25.799174 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:25.799175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:25.799486 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:25.799538 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.799540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.800497 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.800498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.801012 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:25.801014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:25.801480 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:25.801481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:25.801524 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:25.802400 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:25.802405 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:25.804004 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:25.805681 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:25.805691 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:25.805695 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:25.806000 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:25.806227 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:25.806228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:25.806229 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:25.806302 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:25.806531 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:25.806541 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:25.806599 (+ 58us) client_negotiation.cc:769] Sending connection context
0928 14:38:25.806711 (+ 112us) client_negotiation.cc:240] Negotiation successful
0928 14:38:25.806878 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":36,"thread_start_us":95,"threads_started":1}
I20260928 14:38:25.809235 30095 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:43594:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 14:38:25.809495 30095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:38:25.810959 30234 tablet_service.cc:1467] Processing CreateTablet for tablet e349e059ce93492595a6f587b79bf656 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d73b8004354f4b66aa5ee656656f1b62]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:38:25.811074 30374 tablet_service.cc:1467] Processing CreateTablet for tablet e349e059ce93492595a6f587b79bf656 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d73b8004354f4b66aa5ee656656f1b62]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:38:25.811074 30514 tablet_service.cc:1467] Processing CreateTablet for tablet e349e059ce93492595a6f587b79bf656 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d73b8004354f4b66aa5ee656656f1b62]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:38:25.811152 30234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e349e059ce93492595a6f587b79bf656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:25.811226 30374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e349e059ce93492595a6f587b79bf656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:25.811228 30514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e349e059ce93492595a6f587b79bf656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:25.812695 30608 tablet_bootstrap.cc:492] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a: Bootstrap starting.
I20260928 14:38:25.812695 30609 tablet_bootstrap.cc:492] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b: Bootstrap starting.
I20260928 14:38:25.812695 30611 tablet_bootstrap.cc:492] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c: Bootstrap starting.
I20260928 14:38:25.813118 30611 tablet_bootstrap.cc:654] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:25.813124 30609 tablet_bootstrap.cc:654] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:25.813131 30608 tablet_bootstrap.cc:654] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:25.814174 30608 tablet_bootstrap.cc:492] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a: No bootstrap required, opened a new log
I20260928 14:38:25.814245 30608 ts_tablet_manager.cc:1397] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:38:25.814391 30611 tablet_bootstrap.cc:492] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c: No bootstrap required, opened a new log
I20260928 14:38:25.814432 30611 ts_tablet_manager.cc:1397] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 14:38:25.814397 30608 raft_consensus.cc:348] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.814474 30608 raft_consensus.cc:374] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:25.814502 30608 raft_consensus.cc:729] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284ea2ca848a490babfc9c64248c8b7a, State: Initialized, Role: FOLLOWER
I20260928 14:38:25.814555 30608 consensus_queue.cc:260] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.814570 30611 raft_consensus.cc:348] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.814635 30611 raft_consensus.cc:374] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:25.814656 30611 raft_consensus.cc:729] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 715fe73a1af64033b70315b19c0e074c, State: Initialized, Role: FOLLOWER
I20260928 14:38:25.814664 30608 ts_tablet_manager.cc:1428] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:25.814700 30611 consensus_queue.cc:260] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.814713 30628 tablet_replica.cc:439] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } } }
I20260928 14:38:25.814798 30611 ts_tablet_manager.cc:1428] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 14:38:25.814824 30628 tablet_replica.cc:442] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:38:25.814946 30616 tablet_replica.cc:439] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } } }
I20260928 14:38:25.815018 30616 tablet_replica.cc:442] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:38:25.815158 30609 tablet_bootstrap.cc:492] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b: No bootstrap required, opened a new log
I20260928 14:38:25.815209 30609 ts_tablet_manager.cc:1397] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260928 14:38:25.815361 30609 raft_consensus.cc:348] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.815441 30609 raft_consensus.cc:374] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:25.815472 30609 raft_consensus.cc:729] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5563db9576244444bd9ba2c9c492855b, State: Initialized, Role: FOLLOWER
I20260928 14:38:25.815528 30609 consensus_queue.cc:260] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:25.815636 30609 ts_tablet_manager.cc:1428] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:25.816006 30615 tablet_replica.cc:439] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } } }
I20260928 14:38:25.816109 30615 tablet_replica.cc:442] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:38:26.106962 30616 raft_consensus.cc:482] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:38:26.107127 30616 raft_consensus.cc:504] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.107352 30616 leader_election.cc:288] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 284ea2ca848a490babfc9c64248c8b7a (127.22.13.66:37585), 5563db9576244444bd9ba2c9c492855b (127.22.13.67:35827)
I20260928 14:38:26.107635 30394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e349e059ce93492595a6f587b79bf656" candidate_uuid: "715fe73a1af64033b70315b19c0e074c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "284ea2ca848a490babfc9c64248c8b7a" is_pre_election: true
I20260928 14:38:26.107637 30534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e349e059ce93492595a6f587b79bf656" candidate_uuid: "715fe73a1af64033b70315b19c0e074c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5563db9576244444bd9ba2c9c492855b" is_pre_election: true
I20260928 14:38:26.107729 30534 raft_consensus.cc:2454] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 715fe73a1af64033b70315b19c0e074c in term 0.
I20260928 14:38:26.107752 30394 raft_consensus.cc:2454] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 715fe73a1af64033b70315b19c0e074c in term 0.
I20260928 14:38:26.107887 30187 leader_election.cc:302] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5563db9576244444bd9ba2c9c492855b, 715fe73a1af64033b70315b19c0e074c; no voters:
I20260928 14:38:26.108001 30616 raft_consensus.cc:2781] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:38:26.108058 30616 raft_consensus.cc:482] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:38:26.108074 30616 raft_consensus.cc:3037] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:26.108770 30616 raft_consensus.cc:504] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.108914 30616 leader_election.cc:288] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [CANDIDATE]: Term 1 election: Requested vote from peers 284ea2ca848a490babfc9c64248c8b7a (127.22.13.66:37585), 5563db9576244444bd9ba2c9c492855b (127.22.13.67:35827)
I20260928 14:38:26.109112 30394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e349e059ce93492595a6f587b79bf656" candidate_uuid: "715fe73a1af64033b70315b19c0e074c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "284ea2ca848a490babfc9c64248c8b7a"
I20260928 14:38:26.109157 30534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e349e059ce93492595a6f587b79bf656" candidate_uuid: "715fe73a1af64033b70315b19c0e074c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5563db9576244444bd9ba2c9c492855b"
I20260928 14:38:26.109206 30394 raft_consensus.cc:3037] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:26.109217 30534 raft_consensus.cc:3037] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:26.109841 30534 raft_consensus.cc:2454] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 715fe73a1af64033b70315b19c0e074c in term 1.
I20260928 14:38:26.109843 30394 raft_consensus.cc:2454] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 715fe73a1af64033b70315b19c0e074c in term 1.
I20260928 14:38:26.110030 30187 leader_election.cc:302] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5563db9576244444bd9ba2c9c492855b, 715fe73a1af64033b70315b19c0e074c; no voters:
I20260928 14:38:26.110112 30616 raft_consensus.cc:2781] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:26.110168 30616 raft_consensus.cc:686] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [term 1 LEADER]: Becoming Leader. State: Replica: 715fe73a1af64033b70315b19c0e074c, State: Running, Role: LEADER
I20260928 14:38:26.110229 30616 consensus_queue.cc:237] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.110580 30640 tablet_replica.cc:439] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c: TxnStatusTablet state changed. Reason: New leader 715fe73a1af64033b70315b19c0e074c. Latest consensus state: current_term: 1 leader_uuid: "715fe73a1af64033b70315b19c0e074c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } } }
I20260928 14:38:26.110669 30640 tablet_replica.cc:442] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:38:26.110899 30652 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 14:38:26.110886 30097 catalog_manager.cc:5495] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c reported cstate change: term changed from 0 to 1, leader changed from <none> to 715fe73a1af64033b70315b19c0e074c (127.22.13.65). New cstate: current_term: 1 leader_uuid: "715fe73a1af64033b70315b19c0e074c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } health_report { overall_health: UNKNOWN } } }
I20260928 14:38:26.111028 30652 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 14:38:26.173626 30647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:26.170787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:37765 (local address 127.0.0.1:38970)
0928 14:38:26.170850 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:26.170863 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:26.170918 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:26.171159 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:26.171160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:26.171161 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:38:26.171216 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:26.171218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.172094 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.172095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:26.172663 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:26.172664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.173132 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.173132 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:26.173180 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:26.173195 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:38:26.173478 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:38:26.173484 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 14:38:26.173518 (+ 34us) client_negotiation.cc:240] Negotiation successful
0928 14:38:26.173554 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260928 14:38:26.173643 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:26.170934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38970 (local address 127.22.13.65:37765)
0928 14:38:26.171006 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:26.171009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:26.171018 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:38:26.171052 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:26.171053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:26.171072 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:26.171146 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:38:26.171287 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.172007 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.172761 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.173046 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.173128 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:26.173289 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:38:26.173410 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:38:26.173488 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:26.173528 (+ 40us) server_negotiation.cc:299] Negotiation successful
0928 14:38:26.173549 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 14:38:26.174777 30534 raft_consensus.cc:1261] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Refusing update from remote peer 715fe73a1af64033b70315b19c0e074c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:26.174865 30394 raft_consensus.cc:1261] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a [term 1 FOLLOWER]: Refusing update from remote peer 715fe73a1af64033b70315b19c0e074c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:26.175050 30640 consensus_queue.cc:1035] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:26.175182 30616 consensus_queue.cc:1035] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c [LEADER]: Connected to new peer: Peer: permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:26.175848 30615 tablet_replica.cc:439] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b: TxnStatusTablet state changed. Reason: New leader 715fe73a1af64033b70315b19c0e074c. Latest consensus state: current_term: 1 leader_uuid: "715fe73a1af64033b70315b19c0e074c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } } }
I20260928 14:38:26.175953 30615 tablet_replica.cc:442] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:38:26.175959 30628 tablet_replica.cc:439] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a: TxnStatusTablet state changed. Reason: New leader 715fe73a1af64033b70315b19c0e074c. Latest consensus state: current_term: 1 leader_uuid: "715fe73a1af64033b70315b19c0e074c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } } }
I20260928 14:38:26.176054 30628 tablet_replica.cc:442] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:38:26.176753 30616 tablet_replica.cc:439] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "715fe73a1af64033b70315b19c0e074c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } } }
I20260928 14:38:26.176863 30616 tablet_replica.cc:442] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:38:26.177153 30640 tablet_replica.cc:439] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "715fe73a1af64033b70315b19c0e074c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } } }
I20260928 14:38:26.177237 30640 tablet_replica.cc:442] T e349e059ce93492595a6f587b79bf656 P 715fe73a1af64033b70315b19c0e074c: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:38:26.177549 30615 tablet_replica.cc:439] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "715fe73a1af64033b70315b19c0e074c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } } }
I20260928 14:38:26.177654 30615 tablet_replica.cc:442] T e349e059ce93492595a6f587b79bf656 P 5563db9576244444bd9ba2c9c492855b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:38:26.178112 30621 mvcc.cc:205] Tried to move back new op lower bound from 7334323430090072064 to 7334323429827776512. Current Snapshot: MvccSnapshot[applied={T|T < 7334323430090072064}]
I20260928 14:38:26.178200 30628 tablet_replica.cc:439] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "715fe73a1af64033b70315b19c0e074c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } } }
I20260928 14:38:26.178304 30628 tablet_replica.cc:442] T e349e059ce93492595a6f587b79bf656 P 284ea2ca848a490babfc9c64248c8b7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:38:26.181605 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:26.179172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38986 (local address 127.22.13.65:37765)
0928 14:38:26.179210 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:26.179212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:26.179221 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:38:26.179318 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:26.179319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:26.179328 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:26.179388 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:38:26.179484 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.180155 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.180918 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.181192 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.181273 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:26.181373 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:38:26.181406 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:38:26.181467 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:26.181503 (+ 36us) server_negotiation.cc:299] Negotiation successful
0928 14:38:26.181524 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:38:26.186739 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:26.184062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:58871 (local address 127.22.13.65:37765)
0928 14:38:26.184103 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:26.184105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:26.184250 (+ 145us) server_negotiation.cc:407] Connection header received
0928 14:38:26.184304 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:26.184305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:26.184314 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:26.184358 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:38:26.184496 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.184929 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.186055 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.186229 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.186282 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:26.186489 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:38:26.186514 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:38:26.186551 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:26.186647 (+ 96us) server_negotiation.cc:299] Negotiation successful
0928 14:38:26.186667 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":16}
I20260928 14:38:26.186870 30660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:26.183982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:37765 (local address 127.22.13.65:58871)
0928 14:38:26.184173 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:26.184183 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:26.184246 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:26.184362 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:26.184363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:26.184364 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:38:26.184436 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:26.184438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.185012 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.185014 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:26.185908 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:26.185911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.186310 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.186311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:26.186391 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:26.186408 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:38:26.186571 (+ 163us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:38:26.186573 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:38:26.186645 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 14:38:26.186695 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":90,"threads_started":1}
I20260928 14:38:26.189049 30394 raft_consensus.cc:1261] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a [term 1 FOLLOWER]: Refusing update from remote peer 715fe73a1af64033b70315b19c0e074c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:26.189069 30534 raft_consensus.cc:1261] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Refusing update from remote peer 715fe73a1af64033b70315b19c0e074c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:26.189272 30640 consensus_queue.cc:1035] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [LEADER]: Connected to new peer: Peer: permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:26.189410 30657 consensus_queue.cc:1035] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:26.201613 30095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:26.201740 30095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:26.202426 30095 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b: Sending DeleteTablet for 3 replicas of tablet 47d9c245681141d797bb926fd0d6b3ab
I20260928 14:38:26.202747 30514 tablet_service.cc:1514] Processing DeleteTablet for tablet 47d9c245681141d797bb926fd0d6b3ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:26 UTC) from {username='oryx', principal='oryx/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:36018
I20260928 14:38:26.202764 30374 tablet_service.cc:1514] Processing DeleteTablet for tablet 47d9c245681141d797bb926fd0d6b3ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:26 UTC) from {username='oryx', principal='oryx/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:46714
I20260928 14:38:26.202867 30234 tablet_service.cc:1514] Processing DeleteTablet for tablet 47d9c245681141d797bb926fd0d6b3ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:26 UTC) from {username='oryx', principal='oryx/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:38958
I20260928 14:38:26.202885 30633 tablet_replica.cc:330] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b: stopping tablet replica
I20260928 14:38:26.202968 30634 tablet_replica.cc:330] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a: stopping tablet replica
I20260928 14:38:26.202973 30633 raft_consensus.cc:2229] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:26.203034 30634 raft_consensus.cc:2229] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:26.203056 30633 raft_consensus.cc:2258] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:26.203096 30634 raft_consensus.cc:2258] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:26.203176 30632 tablet_replica.cc:330] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c: stopping tablet replica
I20260928 14:38:26.203233 30632 raft_consensus.cc:2229] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:38:26.203347 30632 raft_consensus.cc:2258] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:26.203365 30633 ts_tablet_manager.cc:1905] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:26.203910 30634 ts_tablet_manager.cc:1905] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:26.204042 30632 ts_tablet_manager.cc:1905] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:26.205075 30633 ts_tablet_manager.cc:1918] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:38:26.205137 30633 log.cc:1196] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/47d9c245681141d797bb926fd0d6b3ab
I20260928 14:38:26.205385 30633 ts_tablet_manager.cc:1939] T 47d9c245681141d797bb926fd0d6b3ab P 5563db9576244444bd9ba2c9c492855b: Deleting consensus metadata
I20260928 14:38:26.205490 30634 ts_tablet_manager.cc:1918] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:38:26.205542 30634 log.cc:1196] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/47d9c245681141d797bb926fd0d6b3ab
I20260928 14:38:26.205700 30632 ts_tablet_manager.cc:1918] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:38:26.205751 30632 log.cc:1196] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/wal/wals/47d9c245681141d797bb926fd0d6b3ab
I20260928 14:38:26.205757 30634 ts_tablet_manager.cc:1939] T 47d9c245681141d797bb926fd0d6b3ab P 284ea2ca848a490babfc9c64248c8b7a: Deleting consensus metadata
I20260928 14:38:26.205981 30632 ts_tablet_manager.cc:1939] T 47d9c245681141d797bb926fd0d6b3ab P 715fe73a1af64033b70315b19c0e074c: Deleting consensus metadata
I20260928 14:38:26.206197 30084 catalog_manager.cc:4841] TS 284ea2ca848a490babfc9c64248c8b7a (127.22.13.66:37585): tablet 47d9c245681141d797bb926fd0d6b3ab (table test-table [id=4f0b7fadec234ed6833a3ffbd68e3000]) successfully deleted
I20260928 14:38:26.206290 30081 catalog_manager.cc:4841] TS 715fe73a1af64033b70315b19c0e074c (127.22.13.65:37765): tablet 47d9c245681141d797bb926fd0d6b3ab (table test-table [id=4f0b7fadec234ed6833a3ffbd68e3000]) successfully deleted
I20260928 14:38:26.206310 30083 catalog_manager.cc:4841] TS 5563db9576244444bd9ba2c9c492855b (127.22.13.67:35827): tablet 47d9c245681141d797bb926fd0d6b3ab (table test-table [id=4f0b7fadec234ed6833a3ffbd68e3000]) successfully deleted
Sep 28 14:38:26 dist-test-slave-wwwb krb5kdc[30052](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 14:38:26 dist-test-slave-wwwb krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:26 dist-test-slave-wwwb krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
W20260928 14:38:26.230010 30156 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:26.224237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43596 (local address 127.22.13.126:39087)
0928 14:38:26.224313 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:26.224315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:26.224437 (+ 122us) server_negotiation.cc:407] Connection header received
0928 14:38:26.224527 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:26.224528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:26.224539 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:26.224627 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:26.225232 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.225805 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.226779 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.226983 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.227071 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:26.229904 (+ 2833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43596: BlockingRecv error: recv got EOF from 127.0.0.1:43596 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 28 14:38:26 dist-test-slave-wwwb krb5kdc[30052](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM
I20260928 14:38:26.238477 30156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:26.230959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43606 (local address 127.22.13.126:39087)
0928 14:38:26.231007 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:26.231009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:26.231106 (+ 97us) server_negotiation.cc:407] Connection header received
0928 14:38:26.231181 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:26.231182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:26.231193 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:26.231238 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:26.231726 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.232221 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.232923 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.233153 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.233242 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:26.235756 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:26.235757 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:26.235758 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:26.235766 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:26.237228 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:26.237613 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:26.237615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:26.237615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:26.237679 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:26.237882 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:26.237883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:26.237883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:26.238058 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:26.238180 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:26.238288 (+ 108us) server_negotiation.cc:299] Negotiation successful
0928 14:38:26.238372 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:38:26.240164 30095 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43606:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:38:26.240343 30095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:38:26.241691 30374 tablet_service.cc:1467] Processing CreateTablet for tablet 4cc92dee52f24ad1850725a1fceaac5d (DEFAULT_TABLE table=test-table [id=a5868d2060484a57a196990573aa045a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:26.241701 30234 tablet_service.cc:1467] Processing CreateTablet for tablet 4cc92dee52f24ad1850725a1fceaac5d (DEFAULT_TABLE table=test-table [id=a5868d2060484a57a196990573aa045a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:26.241775 30514 tablet_service.cc:1467] Processing CreateTablet for tablet 4cc92dee52f24ad1850725a1fceaac5d (DEFAULT_TABLE table=test-table [id=a5868d2060484a57a196990573aa045a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:26.241837 30234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cc92dee52f24ad1850725a1fceaac5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:26.241839 30374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cc92dee52f24ad1850725a1fceaac5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:26.241892 30514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cc92dee52f24ad1850725a1fceaac5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:26.243209 30611 tablet_bootstrap.cc:492] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c: Bootstrap starting.
I20260928 14:38:26.243247 30609 tablet_bootstrap.cc:492] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b: Bootstrap starting.
I20260928 14:38:26.243247 30608 tablet_bootstrap.cc:492] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a: Bootstrap starting.
I20260928 14:38:26.243599 30611 tablet_bootstrap.cc:654] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:26.243629 30608 tablet_bootstrap.cc:654] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:26.243652 30609 tablet_bootstrap.cc:654] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:26.244424 30611 tablet_bootstrap.cc:492] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c: No bootstrap required, opened a new log
I20260928 14:38:26.244473 30611 ts_tablet_manager.cc:1397] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:26.244505 30608 tablet_bootstrap.cc:492] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a: No bootstrap required, opened a new log
I20260928 14:38:26.244508 30609 tablet_bootstrap.cc:492] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b: No bootstrap required, opened a new log
I20260928 14:38:26.244550 30609 ts_tablet_manager.cc:1397] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:26.244550 30608 ts_tablet_manager.cc:1397] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:26.244618 30611 raft_consensus.cc:348] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.244685 30611 raft_consensus.cc:374] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:26.244704 30611 raft_consensus.cc:729] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 715fe73a1af64033b70315b19c0e074c, State: Initialized, Role: FOLLOWER
I20260928 14:38:26.244678 30608 raft_consensus.cc:348] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.244746 30608 raft_consensus.cc:374] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:26.244715 30609 raft_consensus.cc:348] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.244743 30611 consensus_queue.cc:260] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.244768 30608 raft_consensus.cc:729] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284ea2ca848a490babfc9c64248c8b7a, State: Initialized, Role: FOLLOWER
I20260928 14:38:26.244781 30609 raft_consensus.cc:374] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:26.244804 30609 raft_consensus.cc:729] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5563db9576244444bd9ba2c9c492855b, State: Initialized, Role: FOLLOWER
I20260928 14:38:26.244820 30611 ts_tablet_manager.cc:1428] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 14:38:26.244813 30608 consensus_queue.cc:260] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.244854 30609 consensus_queue.cc:260] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.244903 30608 ts_tablet_manager.cc:1428] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:26.244957 30609 ts_tablet_manager.cc:1428] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:26.347455 30665 raft_consensus.cc:482] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:38:26.347584 30665 raft_consensus.cc:504] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.347801 30665 leader_election.cc:288] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 284ea2ca848a490babfc9c64248c8b7a (127.22.13.66:37585), 5563db9576244444bd9ba2c9c492855b (127.22.13.67:35827)
I20260928 14:38:26.348048 30534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc92dee52f24ad1850725a1fceaac5d" candidate_uuid: "715fe73a1af64033b70315b19c0e074c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5563db9576244444bd9ba2c9c492855b" is_pre_election: true
I20260928 14:38:26.348065 30394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc92dee52f24ad1850725a1fceaac5d" candidate_uuid: "715fe73a1af64033b70315b19c0e074c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "284ea2ca848a490babfc9c64248c8b7a" is_pre_election: true
I20260928 14:38:26.348168 30534 raft_consensus.cc:2454] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 715fe73a1af64033b70315b19c0e074c in term 0.
I20260928 14:38:26.348187 30394 raft_consensus.cc:2454] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 715fe73a1af64033b70315b19c0e074c in term 0.
I20260928 14:38:26.348338 30187 leader_election.cc:302] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5563db9576244444bd9ba2c9c492855b, 715fe73a1af64033b70315b19c0e074c; no voters:
I20260928 14:38:26.348474 30665 raft_consensus.cc:2781] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:38:26.348508 30665 raft_consensus.cc:482] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:38:26.348538 30665 raft_consensus.cc:3037] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:26.349377 30665 raft_consensus.cc:504] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.349527 30665 leader_election.cc:288] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [CANDIDATE]: Term 1 election: Requested vote from peers 284ea2ca848a490babfc9c64248c8b7a (127.22.13.66:37585), 5563db9576244444bd9ba2c9c492855b (127.22.13.67:35827)
I20260928 14:38:26.349679 30394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc92dee52f24ad1850725a1fceaac5d" candidate_uuid: "715fe73a1af64033b70315b19c0e074c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "284ea2ca848a490babfc9c64248c8b7a"
I20260928 14:38:26.349694 30534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc92dee52f24ad1850725a1fceaac5d" candidate_uuid: "715fe73a1af64033b70315b19c0e074c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5563db9576244444bd9ba2c9c492855b"
I20260928 14:38:26.349776 30394 raft_consensus.cc:3037] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:26.349794 30534 raft_consensus.cc:3037] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:26.350492 30534 raft_consensus.cc:2454] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 715fe73a1af64033b70315b19c0e074c in term 1.
I20260928 14:38:26.350666 30394 raft_consensus.cc:2454] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 715fe73a1af64033b70315b19c0e074c in term 1.
I20260928 14:38:26.350678 30187 leader_election.cc:302] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5563db9576244444bd9ba2c9c492855b, 715fe73a1af64033b70315b19c0e074c; no voters:
I20260928 14:38:26.350780 30665 raft_consensus.cc:2781] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:26.350827 30665 raft_consensus.cc:686] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [term 1 LEADER]: Becoming Leader. State: Replica: 715fe73a1af64033b70315b19c0e074c, State: Running, Role: LEADER
I20260928 14:38:26.350875 30665 consensus_queue.cc:237] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.351529 30095 catalog_manager.cc:5495] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c reported cstate change: term changed from 0 to 1, leader changed from <none> to 715fe73a1af64033b70315b19c0e074c (127.22.13.65). New cstate: current_term: 1 leader_uuid: "715fe73a1af64033b70315b19c0e074c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } health_report { overall_health: UNKNOWN } } }
I20260928 14:38:26.364717 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:26.362638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38992 (local address 127.22.13.65:37765)
0928 14:38:26.362686 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:26.362688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:26.362696 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:38:26.362776 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:26.362777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:26.362786 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:26.362833 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:38:26.362963 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.363417 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.364144 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.364336 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.364385 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:26.364522 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:38:26.364549 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:38:26.364586 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:26.364642 (+ 56us) server_negotiation.cc:299] Negotiation successful
0928 14:38:26.364657 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:38:26.365664 30534 raft_consensus.cc:1261] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Refusing update from remote peer 715fe73a1af64033b70315b19c0e074c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:26.365681 30394 raft_consensus.cc:1261] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a [term 1 FOLLOWER]: Refusing update from remote peer 715fe73a1af64033b70315b19c0e074c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:26.365957 30665 consensus_queue.cc:1035] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:26.366074 30664 consensus_queue.cc:1035] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [LEADER]: Connected to new peer: Peer: permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:26.368637 30095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:26.368727 30095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:26.369367 30095 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b: Sending DeleteTablet for 3 replicas of tablet 4cc92dee52f24ad1850725a1fceaac5d
I20260928 14:38:26.369609 30374 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cc92dee52f24ad1850725a1fceaac5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:26 UTC) from {username='oryx', principal='oryx/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:46714
I20260928 14:38:26.369634 30234 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cc92dee52f24ad1850725a1fceaac5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:26 UTC) from {username='oryx', principal='oryx/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:38958
I20260928 14:38:26.369709 30632 tablet_replica.cc:330] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c: stopping tablet replica
I20260928 14:38:26.369717 30634 tablet_replica.cc:330] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a: stopping tablet replica
I20260928 14:38:26.369721 30514 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cc92dee52f24ad1850725a1fceaac5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:26 UTC) from {username='oryx', principal='oryx/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:36018
I20260928 14:38:26.369753 30632 raft_consensus.cc:2229] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:38:26.369779 30634 raft_consensus.cc:2229] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:26.369796 30633 tablet_replica.cc:330] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b: stopping tablet replica
I20260928 14:38:26.369809 30632 raft_consensus.cc:2258] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:26.369827 30634 raft_consensus.cc:2258] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:26.369841 30633 raft_consensus.cc:2229] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:26.369889 30633 raft_consensus.cc:2258] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:26.369900 30095 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43606:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:38:26.370083 30095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:38:26.370293 30633 ts_tablet_manager.cc:1905] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:26.370393 30634 ts_tablet_manager.cc:1905] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:26.370613 30632 ts_tablet_manager.cc:1905] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:26.371861 30633 ts_tablet_manager.cc:1918] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:38:26.371920 30633 log.cc:1196] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/4cc92dee52f24ad1850725a1fceaac5d
I20260928 14:38:26.371942 30634 ts_tablet_manager.cc:1918] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:38:26.372001 30634 log.cc:1196] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/4cc92dee52f24ad1850725a1fceaac5d
I20260928 14:38:26.372045 30234 tablet_service.cc:1467] Processing CreateTablet for tablet 26946d83fbec424b94621a727d43d6ce (DEFAULT_TABLE table=test-table [id=5cd0753334634e109445381b0e8b2e55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:26.372151 30633 ts_tablet_manager.cc:1939] T 4cc92dee52f24ad1850725a1fceaac5d P 5563db9576244444bd9ba2c9c492855b: Deleting consensus metadata
I20260928 14:38:26.372174 30234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26946d83fbec424b94621a727d43d6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:26.372222 30634 ts_tablet_manager.cc:1939] T 4cc92dee52f24ad1850725a1fceaac5d P 284ea2ca848a490babfc9c64248c8b7a: Deleting consensus metadata
I20260928 14:38:26.372385 30632 ts_tablet_manager.cc:1918] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:38:26.372473 30632 log.cc:1196] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-0/wal/wals/4cc92dee52f24ad1850725a1fceaac5d
I20260928 14:38:26.372671 30632 ts_tablet_manager.cc:1939] T 4cc92dee52f24ad1850725a1fceaac5d P 715fe73a1af64033b70315b19c0e074c: Deleting consensus metadata
I20260928 14:38:26.372931 30084 catalog_manager.cc:4841] TS 284ea2ca848a490babfc9c64248c8b7a (127.22.13.66:37585): tablet 4cc92dee52f24ad1850725a1fceaac5d (table test-table [id=a5868d2060484a57a196990573aa045a]) successfully deleted
I20260928 14:38:26.372956 30081 catalog_manager.cc:4841] TS 715fe73a1af64033b70315b19c0e074c (127.22.13.65:37765): tablet 4cc92dee52f24ad1850725a1fceaac5d (table test-table [id=a5868d2060484a57a196990573aa045a]) successfully deleted
I20260928 14:38:26.373149 30374 tablet_service.cc:1467] Processing CreateTablet for tablet 26946d83fbec424b94621a727d43d6ce (DEFAULT_TABLE table=test-table [id=5cd0753334634e109445381b0e8b2e55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:26.373243 30374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26946d83fbec424b94621a727d43d6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:26.373457 30611 tablet_bootstrap.cc:492] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c: Bootstrap starting.
I20260928 14:38:26.373474 30514 tablet_service.cc:1467] Processing CreateTablet for tablet 26946d83fbec424b94621a727d43d6ce (DEFAULT_TABLE table=test-table [id=5cd0753334634e109445381b0e8b2e55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:26.373561 30514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26946d83fbec424b94621a727d43d6ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:26.373734 30611 tablet_bootstrap.cc:654] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:26.374225 30611 tablet_bootstrap.cc:492] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c: No bootstrap required, opened a new log
I20260928 14:38:26.374289 30611 ts_tablet_manager.cc:1397] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260928 14:38:26.374402 30611 raft_consensus.cc:348] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.374478 30611 raft_consensus.cc:374] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:26.374473 30608 tablet_bootstrap.cc:492] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a: Bootstrap starting.
I20260928 14:38:26.374501 30611 raft_consensus.cc:729] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 715fe73a1af64033b70315b19c0e074c, State: Initialized, Role: FOLLOWER
I20260928 14:38:26.374547 30611 consensus_queue.cc:260] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.374655 30611 ts_tablet_manager.cc:1428] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:26.374781 30608 tablet_bootstrap.cc:654] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:26.374790 30609 tablet_bootstrap.cc:492] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b: Bootstrap starting.
I20260928 14:38:26.375181 30083 catalog_manager.cc:4841] TS 5563db9576244444bd9ba2c9c492855b (127.22.13.67:35827): tablet 4cc92dee52f24ad1850725a1fceaac5d (table test-table [id=a5868d2060484a57a196990573aa045a]) successfully deleted
I20260928 14:38:26.375515 30609 tablet_bootstrap.cc:654] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:26.375653 30608 tablet_bootstrap.cc:492] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a: No bootstrap required, opened a new log
I20260928 14:38:26.375684 30608 ts_tablet_manager.cc:1397] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260928 14:38:26.375756 30608 raft_consensus.cc:348] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.375829 30608 raft_consensus.cc:374] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:26.375854 30608 raft_consensus.cc:729] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284ea2ca848a490babfc9c64248c8b7a, State: Initialized, Role: FOLLOWER
I20260928 14:38:26.375912 30608 consensus_queue.cc:260] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.376022 30608 ts_tablet_manager.cc:1428] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:26.376133 30609 tablet_bootstrap.cc:492] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b: No bootstrap required, opened a new log
I20260928 14:38:26.376206 30609 ts_tablet_manager.cc:1397] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:26.376374 30609 raft_consensus.cc:348] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.376447 30609 raft_consensus.cc:374] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:26.376480 30609 raft_consensus.cc:729] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5563db9576244444bd9ba2c9c492855b, State: Initialized, Role: FOLLOWER
I20260928 14:38:26.376530 30609 consensus_queue.cc:260] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.376627 30609 ts_tablet_manager.cc:1428] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:26.410396 30628 raft_consensus.cc:482] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:38:26.410512 30628 raft_consensus.cc:504] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.410708 30628 leader_election.cc:288] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 715fe73a1af64033b70315b19c0e074c (127.22.13.65:37765), 5563db9576244444bd9ba2c9c492855b (127.22.13.67:35827)
I20260928 14:38:26.411029 30534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26946d83fbec424b94621a727d43d6ce" candidate_uuid: "284ea2ca848a490babfc9c64248c8b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5563db9576244444bd9ba2c9c492855b" is_pre_election: true
I20260928 14:38:26.411029 30254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26946d83fbec424b94621a727d43d6ce" candidate_uuid: "284ea2ca848a490babfc9c64248c8b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "715fe73a1af64033b70315b19c0e074c" is_pre_election: true
I20260928 14:38:26.411147 30534 raft_consensus.cc:2454] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 284ea2ca848a490babfc9c64248c8b7a in term 0.
I20260928 14:38:26.411147 30254 raft_consensus.cc:2454] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 284ea2ca848a490babfc9c64248c8b7a in term 0.
I20260928 14:38:26.411317 30327 leader_election.cc:302] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 284ea2ca848a490babfc9c64248c8b7a, 5563db9576244444bd9ba2c9c492855b; no voters:
I20260928 14:38:26.411489 30628 raft_consensus.cc:2781] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:38:26.411540 30628 raft_consensus.cc:482] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:38:26.411573 30628 raft_consensus.cc:3037] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:26.412529 30628 raft_consensus.cc:504] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.412698 30628 leader_election.cc:288] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [CANDIDATE]: Term 1 election: Requested vote from peers 715fe73a1af64033b70315b19c0e074c (127.22.13.65:37765), 5563db9576244444bd9ba2c9c492855b (127.22.13.67:35827)
I20260928 14:38:26.412849 30254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26946d83fbec424b94621a727d43d6ce" candidate_uuid: "284ea2ca848a490babfc9c64248c8b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "715fe73a1af64033b70315b19c0e074c"
I20260928 14:38:26.412864 30534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26946d83fbec424b94621a727d43d6ce" candidate_uuid: "284ea2ca848a490babfc9c64248c8b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5563db9576244444bd9ba2c9c492855b"
I20260928 14:38:26.412968 30254 raft_consensus.cc:3037] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:26.412964 30534 raft_consensus.cc:3037] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:26.413611 30534 raft_consensus.cc:2454] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 284ea2ca848a490babfc9c64248c8b7a in term 1.
I20260928 14:38:26.413619 30254 raft_consensus.cc:2454] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 284ea2ca848a490babfc9c64248c8b7a in term 1.
I20260928 14:38:26.413823 30325 leader_election.cc:302] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 284ea2ca848a490babfc9c64248c8b7a, 715fe73a1af64033b70315b19c0e074c; no voters:
I20260928 14:38:26.413975 30628 raft_consensus.cc:2781] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:26.414098 30628 raft_consensus.cc:686] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [term 1 LEADER]: Becoming Leader. State: Replica: 284ea2ca848a490babfc9c64248c8b7a, State: Running, Role: LEADER
I20260928 14:38:26.414186 30628 consensus_queue.cc:237] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } }
I20260928 14:38:26.414831 30095 catalog_manager.cc:5495] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a reported cstate change: term changed from 0 to 1, leader changed from <none> to 284ea2ca848a490babfc9c64248c8b7a (127.22.13.66). New cstate: current_term: 1 leader_uuid: "284ea2ca848a490babfc9c64248c8b7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "284ea2ca848a490babfc9c64248c8b7a" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 37585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 } health_report { overall_health: UNKNOWN } } }
I20260928 14:38:26.428104 30691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:26.425592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46742 (local address 127.22.13.66:37585)
0928 14:38:26.425740 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:26.425742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:26.425750 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:38:26.425780 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:26.425781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:26.425790 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:26.425881 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:38:26.426084 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.426566 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.427462 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.427651 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.427706 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:26.427866 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:38:26.427907 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:38:26.427953 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:26.428021 (+ 68us) server_negotiation.cc:299] Negotiation successful
0928 14:38:26.428040 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":102,"threads_started":1}
I20260928 14:38:26.432848 30606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:26.430569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:51843 (local address 127.22.13.65:37765)
0928 14:38:26.430631 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:26.430633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:26.430781 (+ 148us) server_negotiation.cc:407] Connection header received
0928 14:38:26.430825 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:26.430826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:26.430834 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:26.430878 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:38:26.430967 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.431466 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.432260 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.432435 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.432486 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:26.432642 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:38:26.432667 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:38:26.432712 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:26.432775 (+ 63us) server_negotiation.cc:299] Negotiation successful
0928 14:38:26.432790 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:38:26.432876 30694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:26.430506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:37765 (local address 127.22.13.66:51843)
0928 14:38:26.430713 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:26.430724 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:26.430781 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:26.430880 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:26.430880 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:26.430882 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:38:26.430929 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:26.430930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.431535 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.431536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:26.432182 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:26.432184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.432504 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.432505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:26.432558 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:26.432572 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:38:26.432728 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:38:26.432729 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 14:38:26.432771 (+ 42us) client_negotiation.cc:240] Negotiation successful
0928 14:38:26.432810 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":104,"threads_started":1}
I20260928 14:38:26.434918 30254 raft_consensus.cc:1261] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c [term 1 FOLLOWER]: Refusing update from remote peer 284ea2ca848a490babfc9c64248c8b7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:26.434981 30534 raft_consensus.cc:1261] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Refusing update from remote peer 284ea2ca848a490babfc9c64248c8b7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:26.435266 30628 consensus_queue.cc:1035] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5563db9576244444bd9ba2c9c492855b" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 35827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:26.435401 30689 consensus_queue.cc:1035] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "715fe73a1af64033b70315b19c0e074c" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:26.442747 30691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:26.440374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:58169 (local address 127.22.13.66:37585)
0928 14:38:26.440411 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:26.440413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:26.440421 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:38:26.440491 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:26.440492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:26.440501 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:26.440573 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:38:26.440703 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.441190 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.441995 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.442292 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.442361 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:26.442532 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:38:26.442569 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:38:26.442613 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:26.442669 (+ 56us) server_negotiation.cc:299] Negotiation successful
0928 14:38:26.442690 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:38:26.442778 30660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:26.440308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:37585 (local address 127.22.13.65:58169)
0928 14:38:26.440375 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:26.440388 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:26.440434 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:26.440573 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:26.440574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:26.440576 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:38:26.440633 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:26.440635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.441262 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.441263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:26.441860 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:26.441862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:26.442368 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:26.442369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:26.442430 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:26.442438 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:38:26.442628 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:38:26.442630 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:38:26.442668 (+ 38us) client_negotiation.cc:240] Negotiation successful
0928 14:38:26.442718 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260928 14:38:26.448542 30095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:26.448650 30095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:26.449347 30095 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e93b8d2ebe21440a9ec81550a23d398b: Sending DeleteTablet for 3 replicas of tablet 26946d83fbec424b94621a727d43d6ce
I20260928 14:38:26.449566 30234 tablet_service.cc:1514] Processing DeleteTablet for tablet 26946d83fbec424b94621a727d43d6ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:26 UTC) from {username='oryx', principal='oryx/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:38958
I20260928 14:38:26.449611 30374 tablet_service.cc:1514] Processing DeleteTablet for tablet 26946d83fbec424b94621a727d43d6ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:26 UTC) from {username='oryx', principal='oryx/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:46714
I20260928 14:38:26.449671 30632 tablet_replica.cc:330] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c: stopping tablet replica
I20260928 14:38:26.449651 30514 tablet_service.cc:1514] Processing DeleteTablet for tablet 26946d83fbec424b94621a727d43d6ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:26 UTC) from {username='oryx', principal='oryx/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:36018
I20260928 14:38:26.449712 30634 tablet_replica.cc:330] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a: stopping tablet replica
I20260928 14:38:26.449720 30632 raft_consensus.cc:2229] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:26.449753 30632 raft_consensus.cc:2258] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:26.449790 30634 raft_consensus.cc:2229] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:38:26.449817 30633 tablet_replica.cc:330] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b: stopping tablet replica
I20260928 14:38:26.449862 30634 raft_consensus.cc:2258] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:26.449877 30633 raft_consensus.cc:2229] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:26.449909 30633 raft_consensus.cc:2258] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:26.450232 30633 ts_tablet_manager.cc:1905] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:26.450305 30634 ts_tablet_manager.cc:1905] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:26.451790 30633 ts_tablet_manager.cc:1918] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:38:26.451853 30633 log.cc:1196] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/26946d83fbec424b94621a727d43d6ce
I20260928 14:38:26.451874 30634 ts_tablet_manager.cc:1918] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:38:26.451944 30634 log.cc:1196] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/26946d83fbec424b94621a727d43d6ce
I20260928 14:38:26.452131 30633 ts_tablet_manager.cc:1939] T 26946d83fbec424b94621a727d43d6ce P 5563db9576244444bd9ba2c9c492855b: Deleting consensus metadata
I20260928 14:38:26.452190 30634 ts_tablet_manager.cc:1939] T 26946d83fbec424b94621a727d43d6ce P 284ea2ca848a490babfc9c64248c8b7a: Deleting consensus metadata
I20260928 14:38:26.452469 30083 catalog_manager.cc:4841] TS 5563db9576244444bd9ba2c9c492855b (127.22.13.67:35827): tablet 26946d83fbec424b94621a727d43d6ce (table test-table [id=5cd0753334634e109445381b0e8b2e55]) successfully deleted
I20260928 14:38:26.452529 30084 catalog_manager.cc:4841] TS 284ea2ca848a490babfc9c64248c8b7a (127.22.13.66:37585): tablet 26946d83fbec424b94621a727d43d6ce (table test-table [id=5cd0753334634e109445381b0e8b2e55]) successfully deleted
I20260928 14:38:26.452574 30632 ts_tablet_manager.cc:1905] T 26946d83fbec424b94621a727d43d6ce P 715fe73a1af64033b70315b19c0e074c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:26.452785 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 30169
W20260928 14:38:26.460213 30081 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:37765 (error 108)
I20260928 14:38:26.460500 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 30309
I20260928 14:38:26.461251 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:26.460563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:37765 (local address 127.0.0.1:39006)
0928 14:38:26.461177 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:26.461214 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:37765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":37,"threads_started":1}
W20260928 14:38:26.461367 30081 catalog_manager.cc:4584] TS 715fe73a1af64033b70315b19c0e074c (127.22.13.65:37765): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26946d83fbec424b94621a727d43d6ce: Network error: Client connection negotiation failed: client connection to 127.22.13.65:37765: connect: Connection refused (error 111)
I20260928 14:38:26.466490 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 30449
I20260928 14:38:26.471395 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 30065
2026-09-28T14:38:26Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2765 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:38:26 dist-test-slave-wwwb krb5kdc[30702](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:38:26 dist-test-slave-wwwb krb5kdc[30702](info): set up 2 sockets
Sep 28 14:38:26 dist-test-slave-wwwb krb5kdc[30702](info): commencing operation
krb5kdc: starting...
W20260928 14:38:27.498777 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:38:27 dist-test-slave-wwwb krb5kdc[30702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:38:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:38:27.620097 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:37939
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:41769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:37939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:27.701666 30715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:27.701965 30715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:27.702054 30715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:27.703325 30715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:27.703409 30715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:27.703433 30715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:27.703455 30715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:27.703485 30715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:27.705101 30715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:41769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:37939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:37939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:27.705413 30715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:27.705667 30715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:27.710610 30723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:27.710633 30721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:27.710633 30720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:27.710698 30715 server_base.cc:1034] running on GCE node
I20260928 14:38:27.711107 30715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:27.711421 30715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:27.712569 30715 hybrid_clock.cc:648] HybridClock initialized: now 1790606307712548 us; error 37 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-28T14:38:27Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1245 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:38:27 dist-test-slave-wwwb krb5kdc[30728](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:38:27 dist-test-slave-wwwb krb5kdc[30728](info): set up 2 sockets
Sep 28 14:38:27 dist-test-slave-wwwb krb5kdc[30728](info): commencing operation
krb5kdc: starting...
W20260928 14:38:28.746940 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:38:28 dist-test-slave-wwwb krb5kdc[30728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:38:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:38:28.869362 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:42861
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:42861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:28.952795 30741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:28.953033 30741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:28.953058 30741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:28.954293 30741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:28.954339 30741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:28.954353 30741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:28.954366 30741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:28.954378 30741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:28.955914 30741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:42861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:42861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:28.956138 30741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:28.956331 30741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:28.961606 30749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:28.961643 30747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:28.961615 30746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:28.961836 30741 server_base.cc:1034] running on GCE node
I20260928 14:38:28.962222 30741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:28.962595 30741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:28.963750 30741 hybrid_clock.cc:648] HybridClock initialized: now 1790606308963738 us; error 36 us; skew 500 ppm
Sep 28 14:38:28 dist-test-slave-wwwb krb5kdc[30728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606308, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:28.972205 30741 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:38:28.972765 30741 webserver.cc:460] Webserver started at http://127.22.13.126:42859/ using document root <none> and password file <none>
I20260928 14:38:28.973037 30741 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:28.973090 30741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:28.973284 30741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:28.974501 30741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "b459827d8f1e42cf9f8741624a617c4b"
format_stamp: "Formatted at 2026-09-28 14:38:28 on dist-test-slave-wwwb"
server_key: "4bc795cc71419e0e3cf1d23add7c87f2"
server_key_iv: "56f0de4b5db5a78ab66d68e0ca30d08c"
server_key_version: "encryptionkey@0"
I20260928 14:38:28.974908 30741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "b459827d8f1e42cf9f8741624a617c4b"
format_stamp: "Formatted at 2026-09-28 14:38:28 on dist-test-slave-wwwb"
server_key: "4bc795cc71419e0e3cf1d23add7c87f2"
server_key_iv: "56f0de4b5db5a78ab66d68e0ca30d08c"
server_key_version: "encryptionkey@0"
I20260928 14:38:28.976339 30741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:38:28.977144 30756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:28.977337 30741 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:28.977454 30741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "b459827d8f1e42cf9f8741624a617c4b"
format_stamp: "Formatted at 2026-09-28 14:38:28 on dist-test-slave-wwwb"
server_key: "4bc795cc71419e0e3cf1d23add7c87f2"
server_key_iv: "56f0de4b5db5a78ab66d68e0ca30d08c"
server_key_version: "encryptionkey@0"
I20260928 14:38:28.977537 30741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:28.987202 30741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:28.989351 30741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:28.989534 30741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:28.994019 30741 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:42861
I20260928 14:38:28.994076 30808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:42861 every 8 connection(s)
I20260928 14:38:28.994488 30741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:28.995383 30809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:28.997722 30809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b: Bootstrap starting.
I20260928 14:38:28.998358 30809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:28.998698 30809 log.cc:824] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:28.999480 30809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b: No bootstrap required, opened a new log
I20260928 14:38:29.001119 30809 raft_consensus.cc:348] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b459827d8f1e42cf9f8741624a617c4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42861 } }
I20260928 14:38:29.001263 30809 raft_consensus.cc:374] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:29.001286 30809 raft_consensus.cc:729] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b459827d8f1e42cf9f8741624a617c4b, State: Initialized, Role: FOLLOWER
I20260928 14:38:29.001472 30809 consensus_queue.cc:260] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b459827d8f1e42cf9f8741624a617c4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42861 } }
I20260928 14:38:29.001559 30809 raft_consensus.cc:388] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:29.001606 30809 raft_consensus.cc:482] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:29.001659 30809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:29.002241 30809 raft_consensus.cc:504] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b459827d8f1e42cf9f8741624a617c4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42861 } }
I20260928 14:38:29.002357 30809 leader_election.cc:302] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b459827d8f1e42cf9f8741624a617c4b; no voters:
I20260928 14:38:29.002563 30809 leader_election.cc:288] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:29.002617 30812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:29.002792 30812 raft_consensus.cc:686] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 1 LEADER]: Becoming Leader. State: Replica: b459827d8f1e42cf9f8741624a617c4b, State: Running, Role: LEADER
I20260928 14:38:29.002857 30809 sys_catalog.cc:564] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:29.002880 30812 consensus_queue.cc:237] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b459827d8f1e42cf9f8741624a617c4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42861 } }
I20260928 14:38:29.003275 30813 sys_catalog.cc:455] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b459827d8f1e42cf9f8741624a617c4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b459827d8f1e42cf9f8741624a617c4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42861 } } }
I20260928 14:38:29.003366 30813 sys_catalog.cc:458] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:29.003414 30814 sys_catalog.cc:455] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b459827d8f1e42cf9f8741624a617c4b. Latest consensus state: current_term: 1 leader_uuid: "b459827d8f1e42cf9f8741624a617c4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b459827d8f1e42cf9f8741624a617c4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42861 } } }
I20260928 14:38:29.003487 30814 sys_catalog.cc:458] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:29.003592 30819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:29.004173 30819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:29.004626 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 30741
I20260928 14:38:29.004743 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:29.004940 22581 external_mini_cluster.cc:1428] Setting key 61edbfe65b6bb42416dbf810f756add8
I20260928 14:38:29.006228 30819 catalog_manager.cc:1347] Generated new cluster ID: 7e35c26554e14de382030fdb452700c0
I20260928 14:38:29.006291 30819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:38:29 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:29.015587 30831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:29.005441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36126 (local address 127.22.13.126:42861)
0928 14:38:29.005598 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:29.005608 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:29.005644 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:38:29.005800 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:29.005808 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:29.005868 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:29.006812 (+ 944us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:29.006907 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.007838 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.008670 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.009172 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.009311 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:29.011870 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:29.011874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:29.011875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:29.011890 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:29.013796 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:29.014240 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:29.014244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:29.014247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:29.014375 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:29.014609 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:29.014610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:29.014610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:29.014893 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:29.015090 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:29.015197 (+ 107us) server_negotiation.cc:299] Negotiation successful
0928 14:38:29.015355 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260928 14:38:29.046335 30819 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:29.046945 30819 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:29.050995 30819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b: Generated new TSK 0
I20260928 14:38:29.051239 30819 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:38:29.084852 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:42861
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:29.167889 30835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:29.168167 30835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:29.168212 30835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:29.169488 30835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:29.169569 30835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:29.169613 30835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:38:29.171243 30835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:42861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:29.171551 30835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:29.171793 30835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:29.176925 30843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:29.176955 30840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:29.176977 30841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:29.177269 30835 server_base.cc:1034] running on GCE node
I20260928 14:38:29.177507 30835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:29.177809 30835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:29.178992 30835 hybrid_clock.cc:648] HybridClock initialized: now 1790606309178975 us; error 32 us; skew 500 ppm
Sep 28 14:38:29 dist-test-slave-wwwb krb5kdc[30728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606309, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:29.181915 30835 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:38:29.182492 30835 webserver.cc:460] Webserver started at http://127.22.13.65:33489/ using document root <none> and password file <none>
I20260928 14:38:29.182747 30835 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:29.182821 30835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:29.182972 30835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:29.184079 30835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "6e972800f5d8491f95bbb97726992e3e"
format_stamp: "Formatted at 2026-09-28 14:38:29 on dist-test-slave-wwwb"
server_key: "46c1b62bd8f5329797bc1cf944dfd24a"
server_key_iv: "54082884e7757b28b03de2833e038c14"
server_key_version: "encryptionkey@0"
I20260928 14:38:29.184458 30835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "6e972800f5d8491f95bbb97726992e3e"
format_stamp: "Formatted at 2026-09-28 14:38:29 on dist-test-slave-wwwb"
server_key: "46c1b62bd8f5329797bc1cf944dfd24a"
server_key_iv: "54082884e7757b28b03de2833e038c14"
server_key_version: "encryptionkey@0"
I20260928 14:38:29.185853 30835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:29.186697 30850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:29.186869 30835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:29.186977 30835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "6e972800f5d8491f95bbb97726992e3e"
format_stamp: "Formatted at 2026-09-28 14:38:29 on dist-test-slave-wwwb"
server_key: "46c1b62bd8f5329797bc1cf944dfd24a"
server_key_iv: "54082884e7757b28b03de2833e038c14"
server_key_version: "encryptionkey@0"
I20260928 14:38:29.187055 30835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:29.213744 30835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:29.216329 30835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:29.216516 30835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:29.216775 30835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:29.217108 30835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:29.217162 30835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:29.217198 30835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:29.217245 30835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:29.223424 30835 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:36793
I20260928 14:38:29.223471 30963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:36793 every 8 connection(s)
I20260928 14:38:29.223821 30835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
Sep 28 14:38:29 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606309, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:29.230706 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 30835
I20260928 14:38:29.230834 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:38:29.231071 22581 external_mini_cluster.cc:1428] Setting key 6ceb9c01f2df18bdbd9636d36ef5f860
I20260928 14:38:29.234428 30831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:29.224591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:55099 (local address 127.22.13.126:42861)
0928 14:38:29.224655 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:29.224657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:29.224872 (+ 215us) server_negotiation.cc:407] Connection header received
0928 14:38:29.225433 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:29.225434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:29.225450 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:29.225509 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:29.226569 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.227102 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.228153 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.228381 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.228465 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:29.231019 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:29.231021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:29.231021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:29.231031 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:29.232839 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:29.233300 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:29.233302 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:29.233302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:29.233377 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:29.233651 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:29.233652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:29.233653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:29.233783 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:29.233861 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:29.234227 (+ 366us) server_negotiation.cc:299] Negotiation successful
0928 14:38:29.234336 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:38:29.234632 30966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:29.224588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:55099)
0928 14:38:29.224739 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:29.224779 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:29.225315 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:29.225594 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:29.225598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:29.226224 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:29.226442 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:29.226447 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.227217 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.227218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:29.228040 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:29.228043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.228487 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.228488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:29.228624 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:29.229214 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:29.229223 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:29.230902 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:29.232936 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:29.232944 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:29.232946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:29.233202 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:29.233450 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:29.233451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:29.233451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:29.233579 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:29.233851 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:29.233858 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:29.234086 (+ 228us) client_negotiation.cc:769] Sending connection context
0928 14:38:29.234205 (+ 119us) client_negotiation.cc:240] Negotiation successful
0928 14:38:29.234378 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260928 14:38:29.235107 30964 heartbeater.cc:347] Connected to a master server at 127.22.13.126:42861
I20260928 14:38:29.235206 30964 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:29.235415 30964 heartbeater.cc:511] Master 127.22.13.126:42861 requested a full tablet report, sending...
I20260928 14:38:29.235978 30773 ts_manager.cc:194] Registered new tserver with Master: 6e972800f5d8491f95bbb97726992e3e (127.22.13.65:36793)
I20260928 14:38:29.236610 30773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:55099
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:38:29.256395 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:42861
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:29.339643 30969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:29.339866 30969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:29.339902 30969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:29.341207 30969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:29.341259 30969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:29.341318 30969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:38:29.342901 30969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:42861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:29.343168 30969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:29.343421 30969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:29.348603 30975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:29.348596 30977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:29.348596 30974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:29.348618 30969 server_base.cc:1034] running on GCE node
I20260928 14:38:29.348984 30969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:29.349251 30969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:29.350404 30969 hybrid_clock.cc:648] HybridClock initialized: now 1790606309350387 us; error 29 us; skew 500 ppm
Sep 28 14:38:29 dist-test-slave-wwwb krb5kdc[30728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606309, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:29.352950 30969 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:38:29.353447 30969 webserver.cc:460] Webserver started at http://127.22.13.66:33541/ using document root <none> and password file <none>
I20260928 14:38:29.353699 30969 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:29.353770 30969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:29.353916 30969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:29.355044 30969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "1408206df24e473bb3a36260cbb5d58f"
format_stamp: "Formatted at 2026-09-28 14:38:29 on dist-test-slave-wwwb"
server_key: "36994f7119b8d1f63864e862d13ade0a"
server_key_iv: "85967858be2d554c5f888887853b9a70"
server_key_version: "encryptionkey@0"
I20260928 14:38:29.355432 30969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "1408206df24e473bb3a36260cbb5d58f"
format_stamp: "Formatted at 2026-09-28 14:38:29 on dist-test-slave-wwwb"
server_key: "36994f7119b8d1f63864e862d13ade0a"
server_key_iv: "85967858be2d554c5f888887853b9a70"
server_key_version: "encryptionkey@0"
I20260928 14:38:29.356809 30969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:38:29.357574 30984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:29.357764 30969 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:38:29.357872 30969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "1408206df24e473bb3a36260cbb5d58f"
format_stamp: "Formatted at 2026-09-28 14:38:29 on dist-test-slave-wwwb"
server_key: "36994f7119b8d1f63864e862d13ade0a"
server_key_iv: "85967858be2d554c5f888887853b9a70"
server_key_version: "encryptionkey@0"
I20260928 14:38:29.357980 30969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:29.366096 30969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:29.368043 30969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:29.368221 30969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:29.368465 30969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:29.368801 30969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:29.368854 30969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:29.368922 30969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:29.368954 30969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:29.375054 30969 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:33567
I20260928 14:38:29.375097 31097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:33567 every 8 connection(s)
I20260928 14:38:29.375437 30969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
Sep 28 14:38:29 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606309, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:29.382120 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 30969
I20260928 14:38:29.382256 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:38:29.382496 22581 external_mini_cluster.cc:1428] Setting key 1cb3655b3392fbdc124ec248fb10f420
I20260928 14:38:29.386178 30831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:29.376193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:59095 (local address 127.22.13.126:42861)
0928 14:38:29.376253 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:29.376255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:29.376451 (+ 196us) server_negotiation.cc:407] Connection header received
0928 14:38:29.376940 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:29.376941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:29.376950 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:29.377016 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:29.378012 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.378536 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.379536 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.379720 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.379841 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:29.382209 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:29.382210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:29.382211 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:29.382218 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:29.384071 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:29.384673 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:29.384675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:29.384675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:29.384777 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:29.385218 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:29.385219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:29.385220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:29.385386 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:29.385461 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:29.385872 (+ 411us) server_negotiation.cc:299] Negotiation successful
0928 14:38:29.386059 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 14:38:29.386310 31100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:29.376154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:59095)
0928 14:38:29.376317 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:29.376360 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:29.376847 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:29.377077 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:29.377080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:29.377671 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:29.377898 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:29.377904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.378627 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.378628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:29.379454 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:29.379457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.379855 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.379856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:29.380014 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:29.380536 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:29.380545 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:29.382104 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:29.384176 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:29.384184 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:29.384187 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:29.384578 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:29.384897 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:29.384898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:29.384899 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:29.385135 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:29.385505 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:29.385513 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:29.385719 (+ 206us) client_negotiation.cc:769] Sending connection context
0928 14:38:29.385852 (+ 133us) client_negotiation.cc:240] Negotiation successful
0928 14:38:29.386065 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260928 14:38:29.387192 31098 heartbeater.cc:347] Connected to a master server at 127.22.13.126:42861
I20260928 14:38:29.387300 31098 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:29.387519 31098 heartbeater.cc:511] Master 127.22.13.126:42861 requested a full tablet report, sending...
I20260928 14:38:29.388003 30773 ts_manager.cc:194] Registered new tserver with Master: 1408206df24e473bb3a36260cbb5d58f (127.22.13.66:33567)
I20260928 14:38:29.388592 30773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:59095
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:38:29.412274 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:42861
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:29.500013 31103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:29.500243 31103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:29.500269 31103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:29.501428 31103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:29.501469 31103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:29.501518 31103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:38:29.503150 31103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:42861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:29.503454 31103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:29.503751 31103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:29.509739 31108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:29.509801 31111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:29.509763 31109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:29.510087 31103 server_base.cc:1034] running on GCE node
I20260928 14:38:29.510320 31103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:29.510604 31103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:29.511756 31103 hybrid_clock.cc:648] HybridClock initialized: now 1790606309511721 us; error 49 us; skew 500 ppm
Sep 28 14:38:29 dist-test-slave-wwwb krb5kdc[30728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606309, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:29.514267 31103 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:38:29.514775 31103 webserver.cc:460] Webserver started at http://127.22.13.67:39339/ using document root <none> and password file <none>
I20260928 14:38:29.515009 31103 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:29.515084 31103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:29.515240 31103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:29.516331 31103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "152dac752cca485884c5434bdab17249"
format_stamp: "Formatted at 2026-09-28 14:38:29 on dist-test-slave-wwwb"
server_key: "384c43a5828380ece9cd0c4087579abd"
server_key_iv: "a8bea498adc5359fa80b2dca0fad9c94"
server_key_version: "encryptionkey@0"
I20260928 14:38:29.516721 31103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "152dac752cca485884c5434bdab17249"
format_stamp: "Formatted at 2026-09-28 14:38:29 on dist-test-slave-wwwb"
server_key: "384c43a5828380ece9cd0c4087579abd"
server_key_iv: "a8bea498adc5359fa80b2dca0fad9c94"
server_key_version: "encryptionkey@0"
I20260928 14:38:29.518121 31103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:29.518859 31118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:29.519040 31103 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:29.519152 31103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "152dac752cca485884c5434bdab17249"
format_stamp: "Formatted at 2026-09-28 14:38:29 on dist-test-slave-wwwb"
server_key: "384c43a5828380ece9cd0c4087579abd"
server_key_iv: "a8bea498adc5359fa80b2dca0fad9c94"
server_key_version: "encryptionkey@0"
I20260928 14:38:29.519229 31103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:29.527655 31103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:29.530575 31103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:29.530704 31103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:29.530895 31103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:29.531152 31103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:29.531180 31103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:29.531203 31103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:29.531215 31103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:29.536852 31103 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:43067
I20260928 14:38:29.536932 31231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:43067 every 8 connection(s)
I20260928 14:38:29.537279 31103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:38:29.538091 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 31103
I20260928 14:38:29.538228 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:38:29.538445 22581 external_mini_cluster.cc:1428] Setting key 1266698fa8a9aac6c3e7266aad7db097
Sep 28 14:38:29 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606309, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:29.547072 30831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:29.538113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:58921 (local address 127.22.13.126:42861)
0928 14:38:29.538194 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:29.538197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:29.538470 (+ 273us) server_negotiation.cc:407] Connection header received
0928 14:38:29.539145 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:29.539146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:29.539157 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:29.539205 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:29.540276 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.540782 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.541822 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.542091 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.542164 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:29.544325 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:29.544326 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:29.544326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:29.544334 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:29.545663 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:29.546111 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:29.546112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:29.546113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:29.546170 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:29.546432 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:29.546434 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:29.546434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:29.546569 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:29.546652 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:29.546911 (+ 259us) server_negotiation.cc:299] Negotiation successful
0928 14:38:29.546985 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 14:38:29.547220 31234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:29.538129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:58921)
0928 14:38:29.538310 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:29.538364 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:29.539045 (+ 681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:29.539278 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:29.539281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:29.539937 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:29.540161 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:29.540168 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.540889 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.540890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:29.541713 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:29.541715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.542184 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.542185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:29.542323 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:29.542797 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:29.542806 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:29.544227 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:29.545743 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:29.545750 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:29.545752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:29.546042 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:29.546231 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:29.546232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:29.546232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:29.546339 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:29.546652 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:29.546656 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:29.546784 (+ 128us) client_negotiation.cc:769] Sending connection context
0928 14:38:29.546871 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 14:38:29.547007 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
I20260928 14:38:29.547724 31232 heartbeater.cc:347] Connected to a master server at 127.22.13.126:42861
I20260928 14:38:29.547816 31232 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:29.547968 31232 heartbeater.cc:511] Master 127.22.13.126:42861 requested a full tablet report, sending...
I20260928 14:38:29.548316 30773 ts_manager.cc:194] Registered new tserver with Master: 152dac752cca485884c5434bdab17249 (127.22.13.67:43067)
I20260928 14:38:29.548705 30773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:58921
I20260928 14:38:29.550482 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.65.keytab.
Entry for principal oryx/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.65.keytab.
WARNING: no policy specified for oryx/127.22.13.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.66.keytab.
Entry for principal oryx/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.66.keytab.
WARNING: no policy specified for oryx/127.22.13.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.67.keytab.
Entry for principal oryx/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.67.keytab.
I20260928 14:38:29.620558 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 30835
I20260928 14:38:29.627070 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 30969
I20260928 14:38:29.631603 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 31103
I20260928 14:38:29.636122 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 30741
I20260928 14:38:29.641391 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:42861
--webserver_interface=127.22.13.126
--webserver_port=42859
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:42861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:29.724290 31241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:29.724594 31241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:29.724637 31241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:29.725927 31241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:29.726073 31241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:29.726100 31241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:29.726130 31241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:29.726159 31241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:29.727819 31241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:42861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:42861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=42859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:29.728156 31241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:29.728416 31241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:29.733529 31249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:29.733529 31247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:29.733556 31246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:29.734130 31241 server_base.cc:1034] running on GCE node
I20260928 14:38:29.734390 31241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:29.734690 31241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:29.735857 31241 hybrid_clock.cc:648] HybridClock initialized: now 1790606309735845 us; error 40 us; skew 500 ppm
Sep 28 14:38:29 dist-test-slave-wwwb krb5kdc[30728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606309, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:29.738580 31241 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:38:29.739063 31241 webserver.cc:460] Webserver started at http://127.22.13.126:42859/ using document root <none> and password file <none>
I20260928 14:38:29.739279 31241 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:29.739357 31241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:29.740742 31241 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:29.741385 31256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:29.741561 31241 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:29.741660 31241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "b459827d8f1e42cf9f8741624a617c4b"
format_stamp: "Formatted at 2026-09-28 14:38:28 on dist-test-slave-wwwb"
server_key: "4bc795cc71419e0e3cf1d23add7c87f2"
server_key_iv: "56f0de4b5db5a78ab66d68e0ca30d08c"
server_key_version: "encryptionkey@0"
I20260928 14:38:29.742092 31241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:29.782557 31241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:29.784610 31241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:29.784803 31241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:29.789179 31241 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:42861
I20260928 14:38:29.789219 31308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:42861 every 8 connection(s)
I20260928 14:38:29.789633 31241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:29.791232 31309 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:38:29.791864 31309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b: Bootstrap starting.
I20260928 14:38:29.797200 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 31241
I20260928 14:38:29.797588 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:36793
--local_ip_for_outbound_sockets=127.22.13.65
--tserver_master_addrs=127.22.13.126:42861
--webserver_port=33489
--webserver_interface=127.22.13.65
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.13.65
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:38:29.800724 31309 log.cc:824] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:29.802743 31309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:38:29.803066 31309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b: Bootstrap complete.
I20260928 14:38:29.805459 31309 raft_consensus.cc:348] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b459827d8f1e42cf9f8741624a617c4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42861 } }
I20260928 14:38:29.805712 31309 raft_consensus.cc:729] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b459827d8f1e42cf9f8741624a617c4b, State: Initialized, Role: FOLLOWER
I20260928 14:38:29.805892 31309 consensus_queue.cc:260] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b459827d8f1e42cf9f8741624a617c4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42861 } }
I20260928 14:38:29.805985 31309 raft_consensus.cc:388] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:29.806073 31309 raft_consensus.cc:482] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:29.806123 31309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:38:29.807265 31309 raft_consensus.cc:504] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b459827d8f1e42cf9f8741624a617c4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42861 } }
I20260928 14:38:29.807363 31309 leader_election.cc:302] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b459827d8f1e42cf9f8741624a617c4b; no voters:
I20260928 14:38:29.807593 31309 leader_election.cc:288] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 14:38:29.807629 31313 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 2 FOLLOWER]: Leader election won for term 2
I20260928 14:38:29.807753 31313 raft_consensus.cc:686] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [term 2 LEADER]: Becoming Leader. State: Replica: b459827d8f1e42cf9f8741624a617c4b, State: Running, Role: LEADER
I20260928 14:38:29.807847 31313 consensus_queue.cc:237] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b459827d8f1e42cf9f8741624a617c4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42861 } }
I20260928 14:38:29.807911 31309 sys_catalog.cc:564] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:29.808126 31315 sys_catalog.cc:455] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b459827d8f1e42cf9f8741624a617c4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b459827d8f1e42cf9f8741624a617c4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42861 } } }
I20260928 14:38:29.808218 31315 sys_catalog.cc:458] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:29.808465 31314 sys_catalog.cc:455] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b459827d8f1e42cf9f8741624a617c4b. Latest consensus state: current_term: 2 leader_uuid: "b459827d8f1e42cf9f8741624a617c4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b459827d8f1e42cf9f8741624a617c4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42861 } } }
I20260928 14:38:29.808574 31314 sys_catalog.cc:458] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:29.808611 31317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:29.809201 31317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:29.809363 31317 catalog_manager.cc:1259] Loaded cluster ID: 7e35c26554e14de382030fdb452700c0
I20260928 14:38:29.809569 31317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:38:29.810484 31317 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:29.810688 31317 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b459827d8f1e42cf9f8741624a617c4b: Loaded TSK: 0
I20260928 14:38:29.810911 31317 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 14:38:29.897264 31311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:29.897486 31311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:29.897513 31311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:29.898821 31311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:29.898872 31311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:29.898902 31311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:38:29.900427 31311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.65.keytab
--principal=oryx/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:36793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=33489
--tserver_master_addrs=127.22.13.126:42861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:29.900695 31311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:29.900967 31311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:29.906230 31336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:29.906307 31335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:29.906384 31311 server_base.cc:1034] running on GCE node
W20260928 14:38:29.906225 31338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:29.906705 31311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:29.907016 31311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:29.908174 31311 hybrid_clock.cc:648] HybridClock initialized: now 1790606309908151 us; error 39 us; skew 500 ppm
Sep 28 14:38:29 dist-test-slave-wwwb krb5kdc[30728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606309, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:29.910946 31311 init.cc:377] Logged in from keytab as oryx/127.22.13.65@KRBTEST.COM (short username oryx)
I20260928 14:38:29.911499 31311 webserver.cc:460] Webserver started at http://127.22.13.65:33489/ using document root <none> and password file <none>
I20260928 14:38:29.911746 31311 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:29.911806 31311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:29.913276 31311 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:38:29.913954 31345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:29.914126 31311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:29.914237 31311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "6e972800f5d8491f95bbb97726992e3e"
format_stamp: "Formatted at 2026-09-28 14:38:29 on dist-test-slave-wwwb"
server_key: "46c1b62bd8f5329797bc1cf944dfd24a"
server_key_iv: "54082884e7757b28b03de2833e038c14"
server_key_version: "encryptionkey@0"
I20260928 14:38:29.914736 31311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:29.930310 31311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:29.932884 31311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:29.933049 31311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:29.933331 31311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:29.933701 31311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:29.933751 31311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:29.933784 31311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:29.933801 31311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:29.941280 31311 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:36793
I20260928 14:38:29.941314 31458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:36793 every 8 connection(s)
I20260928 14:38:29.941797 31311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:38:29.943573 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 31311
I20260928 14:38:29.943917 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:33567
--local_ip_for_outbound_sockets=127.22.13.66
--tserver_master_addrs=127.22.13.126:42861
--webserver_port=33541
--webserver_interface=127.22.13.66
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.13.66
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:38:29 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:29 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:29.953373 31461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:29.942632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:52865)
0928 14:38:29.942808 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:29.942854 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:29.943449 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:29.944239 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:29.944243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:29.945120 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:29.945446 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:29.945454 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.946798 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.946799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:29.948173 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:29.948177 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.949071 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.949072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:29.949249 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:29.949960 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:29.949972 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:29.953100 (+ 3128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
W20260928 14:38:29.953528 31462 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:29.942712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:52865 (local address 127.22.13.126:42861)
0928 14:38:29.942933 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:29.942942 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:29.942986 (+ 44us) server_negotiation.cc:407] Connection header received
0928 14:38:29.943697 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:29.943705 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:29.943763 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:29.943930 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:29.945600 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.946673 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.948290 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.948945 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.949069 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:29.953292 (+ 4223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:52865: BlockingRecv error: recv got EOF from 127.22.13.65:52865 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":90,"threads_started":1}
W20260928 14:38:29.953605 31459 heartbeater.cc:644] Failed to heartbeat to 127.22.13.126:42861 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.13.126:42861: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Sep 28 14:38:29 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:29 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:29.959781 31461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:29.953855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:56747)
0928 14:38:29.953986 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:29.953998 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:29.954071 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:29.954297 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:29.954298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:29.954474 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:29.954547 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:29.954550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.955488 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.955489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:29.956424 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:29.956427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.956982 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.956983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:29.957054 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:29.957573 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:29.957579 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:29.959702 (+ 2123us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114}
W20260928 14:38:29.960040 31462 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:29.953882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:56747 (local address 127.22.13.126:42861)
0928 14:38:29.953923 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:29.953926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:29.954043 (+ 117us) server_negotiation.cc:407] Connection header received
0928 14:38:29.954188 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:29.954189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:29.954206 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:29.954276 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:29.954638 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.955393 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.956537 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.956897 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.956974 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:29.959816 (+ 2842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:56747: BlockingRecv error: recv got EOF from 127.22.13.65:56747 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 28 14:38:29 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:29 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:29.965623 31461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:29.960050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:60327)
0928 14:38:29.960318 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:29.960328 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:29.960377 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:29.960539 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:29.960540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:29.960688 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:29.960729 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:29.960730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.961624 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.961625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:29.962480 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:29.962482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.962963 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.962964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:29.963030 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:29.963507 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:29.963513 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:29.965539 (+ 2026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":78,"threads_started":1}
W20260928 14:38:29.965716 31462 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:29.960214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:60327 (local address 127.22.13.126:42861)
0928 14:38:29.960247 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:29.960249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:29.960363 (+ 114us) server_negotiation.cc:407] Connection header received
0928 14:38:29.960440 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:29.960441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:29.960452 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:29.960520 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:29.960800 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.961540 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.962569 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:29.962863 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:29.962941 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:29.965649 (+ 2708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:60327: BlockingRecv error: recv got EOF from 127.22.13.65:60327 (error 108)
Metrics: {"server-negotiator.queue_time_us":16,"thread_start_us":64,"threads_started":1}
W20260928 14:38:29.965756 31459 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:38:30.033658 31463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:30.033886 31463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:30.033912 31463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:30.035236 31463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:30.035286 31463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:30.035348 31463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:38:30.036926 31463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.66.keytab
--principal=oryx/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:33567
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=33541
--tserver_master_addrs=127.22.13.126:42861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:30.037210 31463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:30.037464 31463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:30.042709 31470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:30.042716 31471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:30.042994 31463 server_base.cc:1034] running on GCE node
W20260928 14:38:30.042758 31473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:30.043249 31463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:30.043525 31463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:30.044697 31463 hybrid_clock.cc:648] HybridClock initialized: now 1790606310044681 us; error 26 us; skew 500 ppm
Sep 28 14:38:30 dist-test-slave-wwwb krb5kdc[30728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606310, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:30.047421 31463 init.cc:377] Logged in from keytab as oryx/127.22.13.66@KRBTEST.COM (short username oryx)
I20260928 14:38:30.047883 31463 webserver.cc:460] Webserver started at http://127.22.13.66:33541/ using document root <none> and password file <none>
I20260928 14:38:30.048151 31463 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:30.048243 31463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:30.049649 31463 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:38:30.050354 31480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:30.050525 31463 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:30.050637 31463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "1408206df24e473bb3a36260cbb5d58f"
format_stamp: "Formatted at 2026-09-28 14:38:29 on dist-test-slave-wwwb"
server_key: "36994f7119b8d1f63864e862d13ade0a"
server_key_iv: "85967858be2d554c5f888887853b9a70"
server_key_version: "encryptionkey@0"
I20260928 14:38:30.051002 31463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:30.065037 31463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:30.067158 31463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:30.067333 31463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:30.067584 31463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:30.067893 31463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:30.067948 31463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:30.067994 31463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:30.068019 31463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:30.073938 31463 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:33567
I20260928 14:38:30.073999 31593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:33567 every 8 connection(s)
I20260928 14:38:30.074361 31463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:38:30.080476 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 31463
Sep 28 14:38:30 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:30.080821 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:43067
--local_ip_for_outbound_sockets=127.22.13.67
--tserver_master_addrs=127.22.13.126:42861
--webserver_port=39339
--webserver_interface=127.22.13.67
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.13.67
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:38:30 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:30.082557 31596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:30.075075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:59901)
0928 14:38:30.075230 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:30.075266 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:30.075799 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:30.076076 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:30.076079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:30.076662 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:30.076873 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:30.076878 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.077609 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.077610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:30.078523 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:30.078526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.078945 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.078946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:30.079086 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:30.079711 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:30.079719 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:30.082379 (+ 2660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
W20260928 14:38:30.082587 31462 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:30.075076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:59901 (local address 127.22.13.126:42861)
0928 14:38:30.075149 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:30.075150 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:30.075351 (+ 201us) server_negotiation.cc:407] Connection header received
0928 14:38:30.075916 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:30.075917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:30.075945 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:30.075996 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:30.076990 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.077502 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.078621 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.078827 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.078941 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:30.082505 (+ 3564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:59901: BlockingRecv error: recv got EOF from 127.22.13.66:59901 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260928 14:38:30.082896 31594 heartbeater.cc:644] Failed to heartbeat to 127.22.13.126:42861 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.13.126:42861: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Sep 28 14:38:30 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:30 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:30.088209 31596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:30.083177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:55929)
0928 14:38:30.083228 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:30.083240 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:30.083284 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:30.083402 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:30.083403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:30.083574 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:30.083635 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:30.083637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.084274 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.084275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:30.085199 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:30.085202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.085556 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.085557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:30.085642 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:30.086135 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:30.086141 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:30.088115 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 14:38:30.088263 31462 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:30.083240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:55929 (local address 127.22.13.126:42861)
0928 14:38:30.083290 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:30.083292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:30.083319 (+ 27us) server_negotiation.cc:407] Connection header received
0928 14:38:30.083344 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:30.083344 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:30.083354 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:30.083401 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:30.083715 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.084186 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.085288 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.085478 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.085534 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:30.088197 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:55929: BlockingRecv error: recv got EOF from 127.22.13.66:55929 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 28 14:38:30 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:30 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:30.093256 31596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:30.088481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:35745)
0928 14:38:30.088584 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:30.088603 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:30.088643 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:30.088791 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:30.088791 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:30.088900 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:30.088934 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:30.088935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.089900 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.089901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:30.090476 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:30.090478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.090976 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.090976 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:30.091014 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:30.091479 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:30.091484 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:30.093176 (+ 1692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":86}
W20260928 14:38:30.093395 31462 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:30.088545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:35745 (local address 127.22.13.126:42861)
0928 14:38:30.088591 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:30.088593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:30.088634 (+ 41us) server_negotiation.cc:407] Connection header received
0928 14:38:30.088708 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:30.088710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:30.088722 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:30.088782 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:30.089007 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.089820 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.090565 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.090899 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.090980 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:30.093289 (+ 2309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:35745: BlockingRecv error: recv got EOF from 127.22.13.66:35745 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260928 14:38:30.093482 31594 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:38:30.164898 31597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:30.165170 31597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:30.165210 31597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:30.166472 31597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:30.166568 31597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:30.166637 31597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:38:30.168190 31597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/krb5kdc/oryx_127.22.13.67.keytab
--principal=oryx/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:43067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=39339
--tserver_master_addrs=127.22.13.126:42861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:30.168489 31597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:30.168741 31597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:30.174032 31605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:30.174019 31603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:30.174019 31602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:30.174249 31597 server_base.cc:1034] running on GCE node
I20260928 14:38:30.174467 31597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:30.174773 31597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:30.175957 31597 hybrid_clock.cc:648] HybridClock initialized: now 1790606310175947 us; error 38 us; skew 500 ppm
Sep 28 14:38:30 dist-test-slave-wwwb krb5kdc[30728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606310, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:30.178603 31597 init.cc:377] Logged in from keytab as oryx/127.22.13.67@KRBTEST.COM (short username oryx)
I20260928 14:38:30.179145 31597 webserver.cc:460] Webserver started at http://127.22.13.67:39339/ using document root <none> and password file <none>
I20260928 14:38:30.179390 31597 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:30.179478 31597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:30.180891 31597 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:30.181505 31612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:30.181782 31597 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:30.181900 31597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "152dac752cca485884c5434bdab17249"
format_stamp: "Formatted at 2026-09-28 14:38:29 on dist-test-slave-wwwb"
server_key: "384c43a5828380ece9cd0c4087579abd"
server_key_iv: "a8bea498adc5359fa80b2dca0fad9c94"
server_key_version: "encryptionkey@0"
I20260928 14:38:30.182346 31597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:30.190280 31597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:30.192122 31597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:30.192292 31597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:30.192525 31597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:30.192852 31597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:30.192906 31597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:30.192955 31597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:30.192981 31597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:30.199083 31597 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:43067
I20260928 14:38:30.199122 31725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:43067 every 8 connection(s)
I20260928 14:38:30.199487 31597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
Sep 28 14:38:30 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:30 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:30.206687 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 31597
I20260928 14:38:30.207358 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:30.200223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:51405)
0928 14:38:30.200370 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:30.200416 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:30.200917 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:30.201181 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:30.201185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:30.201783 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:30.202079 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:30.202085 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.202792 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.202793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:30.203620 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:30.203622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.204015 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.204016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:30.204158 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:30.204643 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:30.204651 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:30.207110 (+ 2459us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
W20260928 14:38:30.207401 31462 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:30.200252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:51405 (local address 127.22.13.126:42861)
0928 14:38:30.200316 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:30.200319 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:30.200498 (+ 179us) server_negotiation.cc:407] Connection header received
0928 14:38:30.201041 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:30.201043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:30.201055 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:30.201104 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:30.202185 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.202688 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.203720 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.203917 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.204005 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:30.207329 (+ 3324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:51405: BlockingRecv error: recv got EOF from 127.22.13.67:51405 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260928 14:38:30.207782 31726 heartbeater.cc:644] Failed to heartbeat to 127.22.13.126:42861 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.13.126:42861: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Sep 28 14:38:30 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:30 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:30.212689 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:30.208024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:43273)
0928 14:38:30.208065 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:30.208075 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:30.208130 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:30.208236 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:30.208237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:30.208373 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:30.208416 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:30.208418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.209129 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.209130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:30.209692 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:30.209694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.210161 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.210162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:30.210230 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:30.210725 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:30.210729 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:30.212630 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260928 14:38:30.212822 31462 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:30.208077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:43273 (local address 127.22.13.126:42861)
0928 14:38:30.208138 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:30.208140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:30.208148 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:38:30.208183 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:30.208184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:30.208191 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:30.208235 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:30.208509 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.209059 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.209766 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.210092 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.210149 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:30.212747 (+ 2598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:43273: BlockingRecv error: recv got EOF from 127.22.13.67:43273 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 14:38:30.215457 31465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:30.207273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36130 (local address 127.22.13.126:42861)
0928 14:38:30.207346 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:30.207348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:30.207528 (+ 180us) server_negotiation.cc:407] Connection header received
0928 14:38:30.207658 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:30.207659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:30.207671 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:30.207742 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:30.208481 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.209147 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.210070 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.210360 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.210423 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:30.211651 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:30.211656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:30.211657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:30.211671 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:30.213916 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:30.214304 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:30.214308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:30.214308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:30.214443 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:30.214635 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:30.214636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:30.214636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:30.214931 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:30.215086 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:30.215201 (+ 115us) server_negotiation.cc:299] Negotiation successful
0928 14:38:30.215321 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Sep 28 14:38:30 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:30 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:30.217384 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:30.212968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:51177)
0928 14:38:30.213008 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:30.213017 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:30.213050 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:30.213200 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:30.213200 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:30.213328 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:30.213363 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:30.213365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.213989 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.213990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:30.214703 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:30.214705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.215046 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.215047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:30.215102 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:30.215635 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:30.215639 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:30.217299 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260928 14:38:30.217430 31462 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:30.213034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:51177 (local address 127.22.13.126:42861)
0928 14:38:30.213080 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:30.213081 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:30.213089 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:38:30.213109 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:30.213110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:30.213124 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:30.213171 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:30.213438 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.213888 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.214776 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.214955 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.215036 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:30.217371 (+ 2335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:51177: BlockingRecv error: recv got EOF from 127.22.13.67:51177 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260928 14:38:30.217540 31726 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 28 14:38:30 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:30 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:30.970898 31730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:30.966174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:49517)
0928 14:38:30.966313 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:30.966334 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:30.966399 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:30.966525 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:30.966526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:30.966715 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:30.966771 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:30.966773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.967410 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.967411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:30.967964 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:30.967966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.968300 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.968301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:30.968351 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:30.968892 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:30.968897 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:30.970823 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
W20260928 14:38:30.971040 31731 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:30.966255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:49517 (local address 127.22.13.126:42861)
0928 14:38:30.966393 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:30.966395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:30.966403 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:38:30.966449 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:30.966450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:30.966459 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:30.966519 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:30.966837 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.967335 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.968045 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:30.968231 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:30.968280 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:30.970957 (+ 2677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:49517: BlockingRecv error: recv got EOF from 127.22.13.65:49517 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
Sep 28 14:38:31 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:31 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:31.099260 31732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:31.094068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:43485)
0928 14:38:31.094212 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:31.094223 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:31.094308 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:31.094449 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:31.094450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:31.094638 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:31.094687 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:31.094688 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:31.095396 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:31.095398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:31.095952 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:31.095953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:31.096333 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:31.096334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:31.096400 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:31.097091 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:31.097097 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:31.099150 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
W20260928 14:38:31.099313 31731 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:31.094205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:43485 (local address 127.22.13.126:42861)
0928 14:38:31.094288 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:31.094290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:31.094302 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:38:31.094381 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:31.094382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:31.094393 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:31.094445 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:31.094773 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:31.095307 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:31.096060 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:31.096258 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:31.096334 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:31.099246 (+ 2912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:43485: BlockingRecv error: recv got EOF from 127.22.13.66:43485 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 28 14:38:31 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:31 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:31.222999 31733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:31.218167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:37139)
0928 14:38:31.218305 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:31.218317 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:31.218383 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:31.218558 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:31.218559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:31.218740 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:31.218790 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:31.218791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:31.219499 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:31.219501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:31.220120 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:31.220122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:31.220525 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:31.220526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:31.220596 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:31.221053 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:31.221057 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:31.222886 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
W20260928 14:38:31.223042 31731 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:31.218207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:37139 (local address 127.22.13.126:42861)
0928 14:38:31.218266 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:31.218268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:31.218381 (+ 113us) server_negotiation.cc:407] Connection header received
0928 14:38:31.218444 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:31.218446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:31.218461 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:31.218550 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:31.218906 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:31.219406 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:31.220215 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:31.220416 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:31.220511 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:31.222977 (+ 2466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:37139: BlockingRecv error: recv got EOF from 127.22.13.67:37139 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Sep 28 14:38:31 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:31 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:31.977133 31734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:31.971772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:50979)
0928 14:38:31.972009 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:31.972022 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:31.972104 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:31.972283 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:31.972284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:31.972465 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:31.972525 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:31.972527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:31.973203 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:31.973204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:31.973807 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:31.973820 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:31.974389 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:31.974390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:31.974455 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:31.975073 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:31.975079 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:31.977026 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":167,"threads_started":1}
W20260928 14:38:31.977206 31735 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:31.971879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:50979 (local address 127.22.13.126:42861)
0928 14:38:31.972034 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:31.972036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:31.972108 (+ 72us) server_negotiation.cc:407] Connection header received
0928 14:38:31.972179 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:31.972180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:31.972194 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:31.972264 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:31.972603 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:31.973108 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:31.973976 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:31.974317 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:31.974388 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:31.977130 (+ 2742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:50979: BlockingRecv error: recv got EOF from 127.22.13.65:50979 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
Sep 28 14:38:32 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:32 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:32.104642 31736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:32.100005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:37391)
0928 14:38:32.100185 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:32.100197 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:32.100265 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:32.100406 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:32.100407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:32.100572 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:32.100625 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:32.100626 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:32.101254 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:32.101255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:32.101770 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:32.101771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:32.102163 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:32.102164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:32.102256 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:32.102684 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:32.102689 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:32.104580 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":101,"threads_started":1}
W20260928 14:38:32.104832 31735 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:32.100154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:37391 (local address 127.22.13.126:42861)
0928 14:38:32.100242 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:32.100244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:32.100255 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:38:32.100331 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:32.100331 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:32.100342 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:32.100383 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:32.100696 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:32.101184 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:32.101849 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:32.102059 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:32.102153 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:32.104704 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:37391: BlockingRecv error: recv got EOF from 127.22.13.66:37391 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 28 14:38:32 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:32 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:32.228556 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:32.223576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:33669)
0928 14:38:32.223719 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:32.223730 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:32.223782 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:32.224061 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:32.224062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:32.224232 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:32.224280 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:32.224282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:32.224981 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:32.224982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:32.225518 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:32.225520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:32.225899 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:32.225900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:32.225990 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:32.226518 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:32.226524 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:32.228455 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
W20260928 14:38:32.228611 31735 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:32.223844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:33669 (local address 127.22.13.126:42861)
0928 14:38:32.223923 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:32.223925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:32.223932 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:38:32.223955 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:32.223956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:32.223968 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:32.224036 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:32.224367 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:32.224889 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:32.225612 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:32.225826 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:32.225902 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:32.228546 (+ 2644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:33669: BlockingRecv error: recv got EOF from 127.22.13.67:33669 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 28 14:38:32 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:32 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:32.982368 31738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:32.977683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:47831)
0928 14:38:32.977844 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:32.977854 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:32.977917 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:32.978129 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:32.978130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:32.978290 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:32.978343 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:32.978344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:32.978999 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:32.978999 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:32.979537 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:32.979538 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:32.979880 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:32.979880 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:32.979923 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:32.980371 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:32.980375 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:32.982265 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
W20260928 14:38:32.982434 31739 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:32.977826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:47831 (local address 127.22.13.126:42861)
0928 14:38:32.977996 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:32.977998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:32.978014 (+ 16us) server_negotiation.cc:407] Connection header received
0928 14:38:32.978038 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:32.978039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:32.978049 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:32.978110 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:32.978417 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:32.978922 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:32.979619 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:32.979812 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:32.979864 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:32.982359 (+ 2495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:47831: BlockingRecv error: recv got EOF from 127.22.13.65:47831 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
Sep 28 14:38:33 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:33 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:33.110177 31740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:33.105271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:37781)
0928 14:38:33.105535 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:33.105549 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:33.105612 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:33.105776 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:33.105777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:33.106014 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:33.106084 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:33.106085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:33.106789 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:33.106790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:33.107317 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:33.107318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:33.107659 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:33.107659 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:33.107706 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:33.108170 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:33.108174 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:33.110075 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":116,"threads_started":1}
W20260928 14:38:33.110232 31739 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:33.105335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:37781 (local address 127.22.13.126:42861)
0928 14:38:33.105415 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:33.105418 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:33.105616 (+ 198us) server_negotiation.cc:407] Connection header received
0928 14:38:33.105674 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:33.105674 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:33.105687 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:33.105753 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:33.106165 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:33.106717 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:33.107391 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:33.107596 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:33.107645 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:33.110168 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:37781: BlockingRecv error: recv got EOF from 127.22.13.66:37781 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 28 14:38:33 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:33 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:33.233491 31741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:33.229069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:58105)
0928 14:38:33.229223 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:33.229237 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:33.229301 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:33.229439 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:33.229440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:33.229601 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:33.229652 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:33.229653 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:33.230380 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:33.230381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:33.230894 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:33.230895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:33.231205 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:33.231205 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:33.231243 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:33.231664 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:33.231669 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:33.233394 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
W20260928 14:38:33.233551 31739 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:33.229176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:58105 (local address 127.22.13.126:42861)
0928 14:38:33.229225 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:33.229227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:33.229287 (+ 60us) server_negotiation.cc:407] Connection header received
0928 14:38:33.229346 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:33.229347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:33.229360 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:33.229418 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:33.229739 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:33.230309 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:33.230968 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:33.231143 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:33.231191 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:33.233477 (+ 2286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:58105: BlockingRecv error: recv got EOF from 127.22.13.67:58105 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 28 14:38:33 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:33 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:33.991813 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:33.983189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:45077)
0928 14:38:33.983339 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:33.983363 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:33.983437 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:33.983587 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:33.983588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:33.983797 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:33.983862 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:33.983864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:33.984590 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:33.984591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:33.985169 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:33.985171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:33.985578 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:33.985579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:33.985623 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:33.990014 (+ 4391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:33.990021 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:33.991750 (+ 1729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
W20260928 14:38:33.991936 31743 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:33.983249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:45077 (local address 127.22.13.126:42861)
0928 14:38:33.983346 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:33.983348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:33.983454 (+ 106us) server_negotiation.cc:407] Connection header received
0928 14:38:33.983488 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:33.983488 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:33.983500 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:33.983575 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:33.983932 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:33.984523 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:33.985279 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:33.985512 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:33.985565 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:33.991859 (+ 6294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:45077: BlockingRecv error: recv got EOF from 127.22.13.65:45077 (error 108)
Metrics: {"server-negotiator.queue_time_us":69,"thread_start_us":42,"threads_started":1}
Sep 28 14:38:34 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:34 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:34.116003 31744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:34.110710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:52971)
0928 14:38:34.110884 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:34.110895 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:34.110953 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:34.111102 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:34.111102 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:34.111257 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:34.111307 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:34.111308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:34.111968 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:34.111969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:34.112552 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:34.112554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:34.112883 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:34.112884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:34.112930 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:34.113511 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:34.113516 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:34.115912 (+ 2396us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":118,"threads_started":1}
W20260928 14:38:34.116096 31743 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:34.110791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:52971 (local address 127.22.13.126:42861)
0928 14:38:34.110865 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:34.110867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:34.110958 (+ 91us) server_negotiation.cc:407] Connection header received
0928 14:38:34.111004 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:34.111005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:34.111023 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:34.111084 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:34.111371 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:34.111855 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:34.112637 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:34.112810 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:34.112859 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:34.116034 (+ 3175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:52971: BlockingRecv error: recv got EOF from 127.22.13.66:52971 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 28 14:38:34 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:34 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:34.239413 31745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:34.234188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:53061)
0928 14:38:34.234347 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:34.234365 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:34.234418 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:34.234597 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:34.234597 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:34.234776 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:34.234828 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:34.234830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:34.235621 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:34.235623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:34.236289 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:34.236292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:34.236647 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:34.236647 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:34.236687 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:34.237152 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:34.237157 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:34.239356 (+ 2199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
W20260928 14:38:34.239625 31743 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:34.234280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:53061 (local address 127.22.13.126:42861)
0928 14:38:34.234450 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:34.234452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:34.234463 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:38:34.234500 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:34.234501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:34.234516 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:34.234581 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:34.234902 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:34.235529 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:34.236373 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:34.236571 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:34.236627 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:34.239509 (+ 2882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:53061: BlockingRecv error: recv got EOF from 127.22.13.67:53061 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
Sep 28 14:38:34 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:34 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:34.997041 31746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:34.992562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:56921)
0928 14:38:34.992696 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:34.992708 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:34.992760 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:34.992902 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:34.992902 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:34.993068 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:34.993118 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:34.993120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:34.993740 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:34.993741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:34.994311 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:34.994313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:34.994657 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:34.994658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:34.994701 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:34.995131 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:34.995136 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:34.996936 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
W20260928 14:38:34.997098 31747 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:34.992614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:56921 (local address 127.22.13.126:42861)
0928 14:38:34.992739 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:34.992741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:34.992797 (+ 56us) server_negotiation.cc:407] Connection header received
0928 14:38:34.992821 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:34.992821 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:34.992831 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:34.992884 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:34.993182 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:34.993669 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:34.994387 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:34.994590 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:34.994644 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:34.997027 (+ 2383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:56921: BlockingRecv error: recv got EOF from 127.22.13.65:56921 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
Sep 28 14:38:35 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:35 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:35.121536 31748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:35.116678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:49533)
0928 14:38:35.116819 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:35.116833 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:35.116885 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:35.117078 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:35.117079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:35.117233 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:35.117285 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:35.117286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:35.117920 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:35.117921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:35.118597 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:35.118599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:35.119047 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:35.119048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:35.119114 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:35.119670 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:35.119675 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:35.121467 (+ 1792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":60,"threads_started":1}
W20260928 14:38:35.121657 31747 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:35.116865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:49533 (local address 127.22.13.126:42861)
0928 14:38:35.116935 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:35.116937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:35.116947 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:38:35.116978 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:35.116979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:35.117006 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:35.117059 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:35.117347 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:35.117851 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:35.118697 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:35.118953 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:35.119049 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:35.121578 (+ 2529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:49533: BlockingRecv error: recv got EOF from 127.22.13.66:49533 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 28 14:38:35 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:35 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:35.244310 31749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:35.240004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:49135)
0928 14:38:35.240185 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:35.240197 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:35.240275 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:35.240420 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:35.240421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:35.240590 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:35.240641 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:35.240642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:35.241267 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:35.241268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:35.241788 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:35.241789 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:35.242160 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:35.242160 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:35.242199 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:35.242607 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:35.242611 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:35.244234 (+ 1623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":87,"threads_started":1}
W20260928 14:38:35.244345 31747 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:35.240082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:49135 (local address 127.22.13.126:42861)
0928 14:38:35.240183 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:35.240185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:35.240276 (+ 91us) server_negotiation.cc:407] Connection header received
0928 14:38:35.240339 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:35.240340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:35.240351 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:35.240398 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:35.240705 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:35.241197 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:35.241858 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:35.242097 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:35.242151 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:35.244298 (+ 2147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:49135: BlockingRecv error: recv got EOF from 127.22.13.67:49135 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 28 14:38:35 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:36 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:36.002571 31750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:35.997514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:37533)
0928 14:38:35.997704 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:35.997717 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:35.997783 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:35.998037 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:35.998038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:35.998201 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:35.998249 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:35.998250 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:35.999291 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:35.999292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:35.999833 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:35.999835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:36.000190 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:36.000191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:36.000230 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:36.000642 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:36.000646 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:36.002464 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":70,"threads_started":1}
W20260928 14:38:36.002627 31751 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:35.997715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:37533 (local address 127.22.13.126:42861)
0928 14:38:35.997844 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:35.997846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:35.997854 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:38:35.997883 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:35.997883 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:35.997893 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:35.997960 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:35.998333 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:35.999194 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:35.999909 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:36.000125 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:36.000177 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:36.002554 (+ 2377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:37533: BlockingRecv error: recv got EOF from 127.22.13.65:37533 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
Sep 28 14:38:36 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:36 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:36.126777 31752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:36.122140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:51091)
0928 14:38:36.122289 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:36.122300 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:36.122360 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:36.122512 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:36.122513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:36.122674 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:36.122724 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:36.122725 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:36.123346 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:36.123347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:36.123879 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:36.123880 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:36.124199 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:36.124200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:36.124242 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:36.124796 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:36.124801 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:36.126715 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
W20260928 14:38:36.126888 31751 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:36.122223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:51091 (local address 127.22.13.126:42861)
0928 14:38:36.122292 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:36.122298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:36.122358 (+ 60us) server_negotiation.cc:407] Connection header received
0928 14:38:36.122429 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:36.122429 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:36.122440 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:36.122497 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:36.122797 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:36.123281 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:36.123949 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:36.124132 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:36.124182 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:36.126819 (+ 2637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:51091: BlockingRecv error: recv got EOF from 127.22.13.66:51091 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 28 14:38:36 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:36 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:36.249315 31753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:36.244837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:42307)
0928 14:38:36.244962 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:36.244977 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:36.245044 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:36.245150 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:36.245151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:36.245313 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:36.245363 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:36.245364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:36.246117 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:36.246118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:36.246653 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:36.246655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:36.246972 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:36.246973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:36.247022 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:36.247436 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:36.247441 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:36.249255 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
W20260928 14:38:36.249428 31751 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:36.244907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:42307 (local address 127.22.13.126:42861)
0928 14:38:36.244967 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:36.244969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:36.245044 (+ 75us) server_negotiation.cc:407] Connection header received
0928 14:38:36.245085 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:36.245086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:36.245097 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:36.245145 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:36.245420 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:36.246039 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:36.246729 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:36.246911 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:36.246961 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:36.249351 (+ 2390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:42307: BlockingRecv error: recv got EOF from 127.22.13.67:42307 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 28 14:38:37 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:37 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:37.008014 31754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:37.003122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:49435)
0928 14:38:37.003260 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:37.003273 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:37.003353 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:37.003558 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:37.003559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:37.003731 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:37.003781 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:37.003783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:37.004561 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:37.004563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:37.005101 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:37.005103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:37.005506 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:37.005507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:37.005574 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:37.006111 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:37.006115 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:37.007920 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
W20260928 14:38:37.008100 31755 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:37.003163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:49435 (local address 127.22.13.126:42861)
0928 14:38:37.003270 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:37.003272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:37.003354 (+ 82us) server_negotiation.cc:407] Connection header received
0928 14:38:37.003448 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:37.003449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:37.003467 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:37.003540 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:37.003880 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:37.004476 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:37.005193 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:37.005410 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:37.005505 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:37.008024 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:49435: BlockingRecv error: recv got EOF from 127.22.13.65:49435 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
Sep 28 14:38:37 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:37 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:37.131716 31756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:37.127340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:47017)
0928 14:38:37.127484 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:37.127498 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:37.127560 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:37.127718 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:37.127719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:37.127877 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:37.127927 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:37.127928 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:37.128579 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:37.128580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:37.129102 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:37.129104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:37.129426 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:37.129427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:37.129470 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:37.129904 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:37.129908 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:37.131655 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
W20260928 14:38:37.131835 31755 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:37.127422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:47017 (local address 127.22.13.126:42861)
0928 14:38:37.127511 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:37.127512 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:37.127558 (+ 46us) server_negotiation.cc:407] Connection header received
0928 14:38:37.127631 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:37.127632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:37.127643 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:37.127700 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:37.127999 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:37.128511 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:37.129180 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:37.129361 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:37.129411 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:37.131760 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:47017: BlockingRecv error: recv got EOF from 127.22.13.66:47017 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 28 14:38:37 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:37 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:37.254804 31757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:37.249992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:58291)
0928 14:38:37.250135 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:37.250147 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:37.250215 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:37.250381 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:37.250382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:37.250534 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:37.250586 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:37.250588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:37.251231 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:37.251232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:37.251755 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:37.251756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:37.252090 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:37.252091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:37.252131 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:37.252576 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:37.252580 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:37.254707 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
W20260928 14:38:37.254860 31755 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:37.250059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:58291 (local address 127.22.13.126:42861)
0928 14:38:37.250138 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:37.250140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:37.250217 (+ 77us) server_negotiation.cc:407] Connection header received
0928 14:38:37.250260 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:37.250260 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:37.250273 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:37.250346 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:37.250652 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:37.251157 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:37.251828 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:37.252012 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:37.252063 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:37.254788 (+ 2725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:58291: BlockingRecv error: recv got EOF from 127.22.13.67:58291 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 28 14:38:38 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:38 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:38.013034 31758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:38.008484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:51823)
0928 14:38:38.008640 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:38.008651 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:38.008707 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:38.008942 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:38.008943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:38.009118 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:38.009169 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:38.009171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:38.009881 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:38.009882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:38.010461 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:38.010463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:38.010788 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:38.010788 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:38.010828 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:38.011270 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:38.011274 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:38.012958 (+ 1684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
W20260928 14:38:38.013146 31759 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:38.008573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:51823 (local address 127.22.13.126:42861)
0928 14:38:38.008669 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:38.008675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:38.008689 (+ 14us) server_negotiation.cc:407] Connection header received
0928 14:38:38.008804 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:38.008804 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:38.008816 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:38.008889 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:38.009237 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:38.009808 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:38.010535 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:38.010725 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:38.010775 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:38.013069 (+ 2294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:51823: BlockingRecv error: recv got EOF from 127.22.13.65:51823 (error 108)
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":46,"threads_started":1}
Sep 28 14:38:38 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:38 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:38.137328 31760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:38.132434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:54027)
0928 14:38:38.132652 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:38.132664 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:38.132757 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:38.132904 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:38.132905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:38.133064 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:38.133115 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:38.133116 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:38.133892 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:38.133892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:38.134423 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:38.134425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:38.134742 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:38.134743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:38.134782 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:38.135260 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:38.135264 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:38.137258 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":157,"threads_started":1}
W20260928 14:38:38.137400 31759 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:38.132539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:54027 (local address 127.22.13.126:42861)
0928 14:38:38.132592 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:38.132594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:38.132763 (+ 169us) server_negotiation.cc:407] Connection header received
0928 14:38:38.132804 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:38.132804 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:38.132815 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:38.132876 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:38.133179 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:38.133816 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:38.134496 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:38.134677 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:38.134729 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:38.137354 (+ 2625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:54027: BlockingRecv error: recv got EOF from 127.22.13.66:54027 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 28 14:38:38 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:38 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:38.260260 31761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:38.255364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:41655)
0928 14:38:38.255557 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:38.255566 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:38.255628 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:38.255768 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:38.255769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:38.255930 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:38.255990 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:38.255992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:38.256645 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:38.256645 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:38.257176 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:38.257177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:38.257655 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:38.257656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:38.257722 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:38.258265 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:38.258271 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:38.260156 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":104,"threads_started":1}
W20260928 14:38:38.260314 31759 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:38.255425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:41655 (local address 127.22.13.126:42861)
0928 14:38:38.255499 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:38.255501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:38.255634 (+ 133us) server_negotiation.cc:407] Connection header received
0928 14:38:38.255670 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:38.255670 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:38.255684 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:38.255750 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:38.256067 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:38.256569 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:38.257291 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:38.257562 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:38.257630 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:38.260246 (+ 2616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:41655: BlockingRecv error: recv got EOF from 127.22.13.67:41655 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 28 14:38:39 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:39 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:39.024463 31762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:39.013683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:42527)
0928 14:38:39.013869 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:39.013880 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:39.013996 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:39.014149 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:39.014149 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:39.014310 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:39.014362 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:39.014364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:39.015004 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:39.015005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:39.015521 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:39.015522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:39.015848 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:39.015848 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:39.015889 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:39.022509 (+ 6620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:39.022516 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:39.024381 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":113,"threads_started":1}
W20260928 14:38:39.024631 31763 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:39.013819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:42527 (local address 127.22.13.126:42861)
0928 14:38:39.013991 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:39.013993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:39.014003 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:38:39.014058 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:39.014059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:39.014069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:39.014127 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:39.014430 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:39.014923 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:39.015594 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:39.015782 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:39.015833 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:39.024527 (+ 8694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:42527: BlockingRecv error: recv got EOF from 127.22.13.65:42527 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
Sep 28 14:38:39 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:39 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:39.142699 31764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:39.137834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:37277)
0928 14:38:39.138039 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:39.138049 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:39.138130 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:39.138279 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:39.138280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:39.138469 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:39.138518 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:39.138520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:39.139246 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:39.139247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:39.139765 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:39.139767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:39.140172 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:39.140173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:39.140238 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:39.140814 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:39.140819 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:39.142602 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":70,"threads_started":1}
W20260928 14:38:39.142760 31763 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:39.137890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:37277 (local address 127.22.13.126:42861)
0928 14:38:39.137989 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:39.137991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:39.138111 (+ 120us) server_negotiation.cc:407] Connection header received
0928 14:38:39.138208 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:39.138209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:39.138222 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:39.138276 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:39.138611 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:39.139168 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:39.139880 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:39.140083 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:39.140170 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:39.142693 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:37277: BlockingRecv error: recv got EOF from 127.22.13.66:37277 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 28 14:38:39 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:39 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:39.265386 31765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:39.260767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:54759)
0928 14:38:39.260958 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:39.260966 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:39.261041 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:39.261184 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:39.261184 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:39.261367 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:39.261411 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:39.261413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:39.262145 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:39.262146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:39.262678 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:39.262680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:39.263074 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:39.263075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:39.263136 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:39.263553 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:39.263558 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:39.265277 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":98,"threads_started":1}
W20260928 14:38:39.265425 31763 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:39.260834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:54759 (local address 127.22.13.126:42861)
0928 14:38:39.260903 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:39.260905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:39.261024 (+ 119us) server_negotiation.cc:407] Connection header received
0928 14:38:39.261103 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:39.261104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:39.261117 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:39.261176 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:39.261497 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:39.262056 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:39.262777 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:39.262985 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:39.263072 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:39.265360 (+ 2288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:54759: BlockingRecv error: recv got EOF from 127.22.13.67:54759 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 28 14:38:40 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:40 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:40.029981 31766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:40.025211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:41135)
0928 14:38:40.025390 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:40.025402 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:40.025457 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:40.025610 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:40.025611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:40.025768 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:40.025821 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:40.025822 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:40.026493 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:40.026493 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:40.027022 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:40.027023 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:40.027361 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:40.027362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:40.027403 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:40.027842 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:40.027847 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:40.029847 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
W20260928 14:38:40.030037 31767 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:40.025317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:41135 (local address 127.22.13.126:42861)
0928 14:38:40.025452 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:40.025454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:40.025464 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:38:40.025511 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:40.025511 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:40.025521 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:40.025592 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:40.025886 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:40.026409 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:40.027095 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:40.027293 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:40.027345 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:40.029929 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:41135: BlockingRecv error: recv got EOF from 127.22.13.65:41135 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
Sep 28 14:38:40 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:40 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:40.147738 31768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:40.143173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:54255)
0928 14:38:40.143306 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:40.143316 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:40.143370 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:40.143504 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:40.143504 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:40.143653 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:40.143702 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:40.143704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:40.144329 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:40.144330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:40.144913 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:40.144915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:40.145256 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:40.145257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:40.145299 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:40.145806 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:40.145811 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:40.147638 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
W20260928 14:38:40.147792 31767 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:40.143237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:54255 (local address 127.22.13.126:42861)
0928 14:38:40.143300 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:40.143302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:40.143374 (+ 72us) server_negotiation.cc:407] Connection header received
0928 14:38:40.143422 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:40.143423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:40.143432 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:40.143490 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:40.143760 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:40.144262 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:40.144992 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:40.145189 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:40.145244 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:40.147727 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:54255: BlockingRecv error: recv got EOF from 127.22.13.66:54255 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Sep 28 14:38:40 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:40 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:40.270388 31769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:40.265884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:36421)
0928 14:38:40.266070 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:40.266083 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:40.266145 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:40.266302 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:40.266302 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:40.266477 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:40.266544 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:40.266546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:40.267181 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:40.267182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:40.267718 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:40.267720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:40.268059 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:40.268060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:40.268101 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:40.268540 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:40.268545 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:40.270330 (+ 1785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":133,"threads_started":1}
W20260928 14:38:40.270501 31767 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:40.266103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:36421 (local address 127.22.13.126:42861)
0928 14:38:40.266154 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:40.266156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:40.266171 (+ 15us) server_negotiation.cc:407] Connection header received
0928 14:38:40.266197 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:40.266197 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:40.266210 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:40.266281 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:40.266620 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:40.267111 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:40.267795 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:40.267976 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:40.268026 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:40.270430 (+ 2404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:36421: BlockingRecv error: recv got EOF from 127.22.13.67:36421 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 28 14:38:41 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:41 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:41.035316 31770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:41.030535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:45803)
0928 14:38:41.030711 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:41.030722 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:41.030792 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:41.030937 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:41.030938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:41.031101 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:41.031149 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:41.031151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:41.031806 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:41.031806 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:41.032342 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:41.032343 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:41.032673 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:41.032674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:41.032715 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:41.033267 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:41.033272 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:41.035202 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":110,"threads_started":1}
W20260928 14:38:41.035365 31771 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:41.030629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:45803 (local address 127.22.13.126:42861)
0928 14:38:41.030731 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:41.030733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:41.030779 (+ 46us) server_negotiation.cc:407] Connection header received
0928 14:38:41.030861 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:41.030862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:41.030872 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:41.030920 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:41.031216 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:41.031731 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:41.032418 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:41.032608 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:41.032659 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:41.035290 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:45803: BlockingRecv error: recv got EOF from 127.22.13.65:45803 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":51,"threads_started":1}
Sep 28 14:38:41 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:41 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:41.152676 31772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:41.148254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:56443)
0928 14:38:41.148381 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:41.148391 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:41.148448 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:41.148590 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:41.148591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:41.148760 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:41.148808 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:41.148810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:41.149542 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:41.149542 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:41.150086 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:41.150087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:41.150392 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:41.150392 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:41.150448 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:41.150866 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:41.150870 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:41.152608 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
W20260928 14:38:41.152808 31771 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:41.148297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:56443 (local address 127.22.13.126:42861)
0928 14:38:41.148364 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:41.148366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:41.148453 (+ 87us) server_negotiation.cc:407] Connection header received
0928 14:38:41.148496 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:41.148496 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:41.148506 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:41.148555 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:41.148867 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:41.149471 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:41.150150 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:41.150329 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:41.150380 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:41.152729 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:56443: BlockingRecv error: recv got EOF from 127.22.13.66:56443 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 28 14:38:41 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:41 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:41.275424 31773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:41.271095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:34901)
0928 14:38:41.271218 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:41.271229 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:41.271282 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:41.271406 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:41.271407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:41.271578 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:41.271628 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:41.271629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:41.272241 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:41.272241 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:41.272800 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:41.272802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:41.273129 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:41.273130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:41.273170 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:41.273614 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:41.273619 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:41.275356 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":67,"threads_started":1}
W20260928 14:38:41.275509 31771 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:41.271163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:34901 (local address 127.22.13.126:42861)
0928 14:38:41.271245 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:41.271247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:41.271271 (+ 24us) server_negotiation.cc:407] Connection header received
0928 14:38:41.271343 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:41.271343 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:41.271356 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:41.271392 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:41.271686 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:41.272171 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:41.272877 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:41.273061 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:41.273111 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:41.275459 (+ 2348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:34901: BlockingRecv error: recv got EOF from 127.22.13.67:34901 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 28 14:38:42 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:42 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:42.040638 31774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:42.035936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:37319)
0928 14:38:42.036085 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:42.036097 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:42.036150 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:42.036316 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:42.036317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:42.036494 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:42.036546 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:42.036547 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:42.037305 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:42.037306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:42.038088 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:42.038090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:42.038434 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:42.038435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:42.038474 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:42.038922 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:42.038927 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:42.040580 (+ 1653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
W20260928 14:38:42.040760 31775 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:42.036029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:37319 (local address 127.22.13.126:42861)
0928 14:38:42.036168 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:42.036177 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:42.036187 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:38:42.036213 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:42.036214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:42.036224 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:42.036298 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:42.036622 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:42.037181 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:42.038156 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:42.038360 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:42.038420 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:42.040678 (+ 2258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:37319: BlockingRecv error: recv got EOF from 127.22.13.65:37319 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
Sep 28 14:38:42 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:42 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:42.158056 31776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:42.153260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:43417)
0928 14:38:42.153468 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:42.153483 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:42.153559 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:42.153718 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:42.153718 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:42.153900 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:42.153987 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:42.153989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:42.154695 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:42.154696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:42.155251 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:42.155253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:42.155641 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:42.155642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:42.155701 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:42.156197 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:42.156202 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:42.157942 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":97,"threads_started":1}
W20260928 14:38:42.158133 31775 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:42.153372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:43417 (local address 127.22.13.126:42861)
0928 14:38:42.153423 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:42.153425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:42.153540 (+ 115us) server_negotiation.cc:407] Connection header received
0928 14:38:42.153625 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:42.153626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:42.153637 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:42.153711 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:42.154052 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:42.154616 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:42.155339 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:42.155552 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:42.155639 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:42.158070 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:43417: BlockingRecv error: recv got EOF from 127.22.13.66:43417 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 28 14:38:42 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:42 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:42.280797 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:42.276029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:50583)
0928 14:38:42.276187 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:42.276202 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:42.276260 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:42.276401 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:42.276402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:42.276574 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:42.276625 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:42.276626 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:42.277265 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:42.277265 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:42.277802 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:42.277804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:42.278218 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:42.278219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:42.278264 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:42.278700 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:42.278705 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:42.280736 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
W20260928 14:38:42.280936 31775 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:42.276139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:50583 (local address 127.22.13.126:42861)
0928 14:38:42.276207 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:42.276209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:42.276249 (+ 40us) server_negotiation.cc:407] Connection header received
0928 14:38:42.276308 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:42.276308 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:42.276320 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:42.276381 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:42.276699 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:42.277192 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:42.277875 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:42.278153 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:42.278208 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:42.280831 (+ 2623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:50583: BlockingRecv error: recv got EOF from 127.22.13.67:50583 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 28 14:38:43 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:43 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:43.046250 31778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:43.041343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:35871)
0928 14:38:43.041525 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:43.041535 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:43.041594 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:43.041730 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:43.041731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:43.041894 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:43.042003 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:43.042005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:43.042634 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:43.042634 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:43.043156 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:43.043158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:43.043650 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:43.043651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:43.043714 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:43.044144 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:43.044149 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:43.046178 (+ 2029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":117,"threads_started":1}
W20260928 14:38:43.046352 31779 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:43.041461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:35871 (local address 127.22.13.126:42861)
0928 14:38:43.041595 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:43.041597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:43.041626 (+ 29us) server_negotiation.cc:407] Connection header received
0928 14:38:43.041651 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:43.041651 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:43.041661 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:43.041709 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:43.042078 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:43.042567 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:43.043243 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:43.043564 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:43.043629 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:43.046280 (+ 2651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:35871: BlockingRecv error: recv got EOF from 127.22.13.65:35871 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":54,"threads_started":1}
Sep 28 14:38:43 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:43 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:43.163314 31780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:43.158662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:39409)
0928 14:38:43.158789 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:43.158808 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:43.158911 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:43.159075 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:43.159076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:43.159256 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:43.159313 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:43.159315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:43.160015 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:43.160017 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:43.160550 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:43.160552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:43.160920 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:43.160921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:43.160986 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:43.161502 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:43.161506 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:43.163202 (+ 1696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
W20260928 14:38:43.163352 31779 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:43.158805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:39409 (local address 127.22.13.126:42861)
0928 14:38:43.158870 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:43.158872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:43.158908 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:38:43.158987 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:43.158988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:43.159007 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:43.159070 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:43.159404 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:43.159903 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:43.160634 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:43.160833 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:43.160924 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:43.163287 (+ 2363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:39409: BlockingRecv error: recv got EOF from 127.22.13.66:39409 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 28 14:38:43 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:43 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:43.286253 31781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:43.281471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:60091)
0928 14:38:43.281625 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:43.281643 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:43.281701 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:43.281843 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:43.281844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:43.282029 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:43.282079 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:43.282081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:43.282725 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:43.282725 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:43.283246 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:43.283247 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:43.283588 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:43.283589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:43.283630 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:43.284185 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:43.284191 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:43.286125 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":100,"threads_started":1}
W20260928 14:38:43.286266 31779 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:43.281546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:60091 (local address 127.22.13.126:42861)
0928 14:38:43.281630 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:43.281633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:43.281702 (+ 69us) server_negotiation.cc:407] Connection header received
0928 14:38:43.281748 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:43.281749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:43.281761 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:43.281824 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:43.282146 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:43.282652 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:43.283327 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:43.283519 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:43.283573 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:43.286213 (+ 2640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:60091: BlockingRecv error: recv got EOF from 127.22.13.67:60091 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 28 14:38:44 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:44 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:44.056318 31782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:44.046735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:43709)
0928 14:38:44.046886 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:44.046896 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:44.046951 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:44.047084 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:44.047085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:44.047254 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:44.047308 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:44.047309 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:44.047948 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:44.047948 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:44.048488 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:44.048490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:44.048819 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:44.048819 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:44.048859 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:44.054327 (+ 5468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:44.054332 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:44.056258 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
W20260928 14:38:44.056469 31783 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:44.046812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:43709 (local address 127.22.13.126:42861)
0928 14:38:44.046954 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:44.046956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:44.046984 (+ 28us) server_negotiation.cc:407] Connection header received
0928 14:38:44.047006 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:44.047006 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:44.047017 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:44.047065 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:44.047378 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:44.047879 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:44.048565 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:44.048754 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:44.048805 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:44.056368 (+ 7563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:43709: BlockingRecv error: recv got EOF from 127.22.13.65:43709 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
Sep 28 14:38:44 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:44 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:44.168164 31784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:44.163884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:48831)
0928 14:38:44.164057 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:44.164067 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:44.164118 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:44.164262 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:44.164263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:44.164423 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:44.164473 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:44.164474 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:44.165072 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:44.165072 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:44.165595 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:44.165596 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:44.165908 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:44.165909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:44.166005 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:44.166417 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:44.166421 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:44.168105 (+ 1684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":95,"threads_started":1}
W20260928 14:38:44.168305 31783 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:44.163967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:48831 (local address 127.22.13.126:42861)
0928 14:38:44.164023 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:44.164025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:44.164117 (+ 92us) server_negotiation.cc:407] Connection header received
0928 14:38:44.164187 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:44.164187 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:44.164198 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:44.164241 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:44.164532 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:44.165004 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:44.165661 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:44.165840 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:44.165889 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:44.168228 (+ 2339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:48831: BlockingRecv error: recv got EOF from 127.22.13.66:48831 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Sep 28 14:38:44 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:44 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.67@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:44.291610 31785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:44.286708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.67:60861)
0928 14:38:44.286895 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:44.286919 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:44.287014 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:44.287184 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:44.287185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:44.287379 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:44.287429 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:44.287431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:44.288131 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:44.288132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:44.288686 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:44.288688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:44.289066 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:44.289068 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:44.289125 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:44.289580 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:44.289585 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:44.291539 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":103,"threads_started":1}
W20260928 14:38:44.291720 31783 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:44.286798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:60861 (local address 127.22.13.126:42861)
0928 14:38:44.286879 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:44.286881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:44.286994 (+ 113us) server_negotiation.cc:407] Connection header received
0928 14:38:44.287085 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:44.287086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:44.287106 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:44.287173 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:44.287524 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:44.288025 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:44.288779 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:44.288976 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:44.289065 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:44.291648 (+ 2583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.67:60861: BlockingRecv error: recv got EOF from 127.22.13.67:60861 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 28 14:38:45 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:45 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.65@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:45.061100 31786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:45.056898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.65:57585)
0928 14:38:45.057038 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:45.057048 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:45.057095 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:45.057240 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:45.057240 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:45.057385 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:45.057436 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:45.057438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.058076 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.058077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:45.058589 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:45.058591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.058914 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.058915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:45.058956 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:45.059352 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:45.059357 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:45.061041 (+ 1684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
W20260928 14:38:45.061254 31787 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:45.057000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:57585 (local address 127.22.13.126:42861)
0928 14:38:45.057130 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:45.057132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:45.057139 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:38:45.057161 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:45.057162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:45.057171 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:45.057230 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:45.057499 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.058009 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.058663 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.058846 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.058900 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:45.061161 (+ 2261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.65:57585: BlockingRecv error: recv got EOF from 127.22.13.65:57585 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
Sep 28 14:38:45 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:38:45 dist-test-slave-wwwb krb5kdc[30728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.13.66@KRBTEST.COM for oryx/127.22.13.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:38:45.173169 31788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:45.168799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42861 (local address 127.22.13.66:47709)
0928 14:38:45.168959 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:45.168970 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:45.169019 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:45.169220 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:45.169220 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:45.169375 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:45.169433 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:45.169434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.170067 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.170068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:45.170583 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:45.170585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.170906 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.170907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:45.170945 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:45.171363 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:45.171367 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:45.173064 (+ 1697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.13.126:42861: Server oryx/127.22.13.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
W20260928 14:38:45.173264 31787 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:45.169014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:47709 (local address 127.22.13.126:42861)
0928 14:38:45.169077 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:45.169079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:45.169091 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:38:45.169127 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:45.169128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:45.169138 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:45.169203 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:45.169497 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.169999 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.170660 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.170841 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.170902 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:45.173193 (+ 2291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.13.66:47709: BlockingRecv error: recv got EOF from 127.22.13.66:47709 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:38:45.216534 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 31311
I20260928 14:38:45.221161 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 31463
I20260928 14:38:45.225569 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 31597
I20260928 14:38:45.229863 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 31241
2026-09-28T14:38:45Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17511 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-28T14:38:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:45Z Disabled control of system clock
I20260928 14:38:45.257558 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:46027
--webserver_interface=127.22.13.126
--webserver_port=0
--builtin_ntp_servers=127.22.13.84:36991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:46027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260928 14:38:45.337173 31792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:45.337391 31792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:45.337428 31792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:45.338573 31792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:45.338611 31792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:45.338624 31792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:45.338636 31792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:45.338647 31792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:45.339902 31792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:46027
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:46027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:45.340094 31792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:45.340319 31792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:38:45.345573 31792 server_base.cc:1034] running on GCE node
W20260928 14:38:45.345564 31797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:45.345547 31798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:45.345546 31800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:45.346133 31792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:45.346477 31792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:45.347626 31792 hybrid_clock.cc:648] HybridClock initialized: now 1790606325347608 us; error 41 us; skew 500 ppm
I20260928 14:38:45.348918 31792 webserver.cc:460] Webserver started at http://127.22.13.126:39311/ using document root <none> and password file <none>
I20260928 14:38:45.349124 31792 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:45.349166 31792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:45.349256 31792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:45.350415 31792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "86bf0132de514c04af989e717247644b"
format_stamp: "Formatted at 2026-09-28 14:38:45 on dist-test-slave-wwwb"
server_key: "53b8138d6235f7678b7728b5e0fc8e69"
server_key_iv: "821bac8807eaae96b2621a3ebeabc5c4"
server_key_version: "encryptionkey@0"
I20260928 14:38:45.350718 31792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "86bf0132de514c04af989e717247644b"
format_stamp: "Formatted at 2026-09-28 14:38:45 on dist-test-slave-wwwb"
server_key: "53b8138d6235f7678b7728b5e0fc8e69"
server_key_iv: "821bac8807eaae96b2621a3ebeabc5c4"
server_key_version: "encryptionkey@0"
I20260928 14:38:45.351965 31792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:38:45.352645 31806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:45.352824 31792 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:45.352921 31792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "86bf0132de514c04af989e717247644b"
format_stamp: "Formatted at 2026-09-28 14:38:45 on dist-test-slave-wwwb"
server_key: "53b8138d6235f7678b7728b5e0fc8e69"
server_key_iv: "821bac8807eaae96b2621a3ebeabc5c4"
server_key_version: "encryptionkey@0"
I20260928 14:38:45.352981 31792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:45.363827 31792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:45.364116 31792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:45.364219 31792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:45.368091 31792 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:46027
I20260928 14:38:45.368170 31858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:46027 every 8 connection(s)
I20260928 14:38:45.368525 31792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:45.369252 31859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:45.371939 31859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b: Bootstrap starting.
I20260928 14:38:45.372288 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 31792
I20260928 14:38:45.372419 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:45.372642 22581 external_mini_cluster.cc:1428] Setting key 799239a7481fdd4da15d029fcad6a443
I20260928 14:38:45.372780 31859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:45.373108 31859 log.cc:824] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:45.373919 31859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b: No bootstrap required, opened a new log
I20260928 14:38:45.375599 31859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86bf0132de514c04af989e717247644b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46027 } }
I20260928 14:38:45.375783 31859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:45.375820 31859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86bf0132de514c04af989e717247644b, State: Initialized, Role: FOLLOWER
I20260928 14:38:45.375954 31859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86bf0132de514c04af989e717247644b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46027 } }
I20260928 14:38:45.376286 31859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:45.376322 31859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:45.376382 31859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:45.377118 31859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86bf0132de514c04af989e717247644b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46027 } }
I20260928 14:38:45.377223 31859 leader_election.cc:302] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86bf0132de514c04af989e717247644b; no voters:
I20260928 14:38:45.377460 31859 leader_election.cc:288] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:45.377485 31864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:45.377593 31864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [term 1 LEADER]: Becoming Leader. State: Replica: 86bf0132de514c04af989e717247644b, State: Running, Role: LEADER
I20260928 14:38:45.377686 31864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86bf0132de514c04af989e717247644b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46027 } }
I20260928 14:38:45.377753 31859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:45.378115 31864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86bf0132de514c04af989e717247644b. Latest consensus state: current_term: 1 leader_uuid: "86bf0132de514c04af989e717247644b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86bf0132de514c04af989e717247644b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46027 } } }
I20260928 14:38:45.378294 31864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:45.378129 31866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86bf0132de514c04af989e717247644b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86bf0132de514c04af989e717247644b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46027 } } }
I20260928 14:38:45.378582 31866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:45.378686 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:45.373240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55136 (local address 127.22.13.126:46027)
0928 14:38:45.373408 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:45.373415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:45.373439 (+ 24us) server_negotiation.cc:407] Connection header received
0928 14:38:45.373869 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:45.373875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:45.373989 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:45.374171 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:45.374400 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.375266 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.376433 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.377051 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.377179 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:45.377959 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:45.377963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:45.377963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:45.378168 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:45.378210 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:45.378224 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:45.378290 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:45.378365 (+ 75us) server_negotiation.cc:299] Negotiation successful
0928 14:38:45.378453 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
I20260928 14:38:45.378718 31872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:45.379220 31872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:45.380991 31872 catalog_manager.cc:1347] Generated new cluster ID: 40a6ac203b8a4303989b362cd85a93c4
I20260928 14:38:45.381071 31872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:38:45.395514 31872 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:45.396288 31872 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:45.404636 31872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86bf0132de514c04af989e717247644b: Generated new TSK 0
I20260928 14:38:45.404873 31872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:38:45.409487 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46027
--builtin_ntp_servers=127.22.13.84:36991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260928 14:38:45.492610 31883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:45.492826 31883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:45.492867 31883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:45.494165 31883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:45.494205 31883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:45.494232 31883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:38:45.495640 31883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:45.495906 31883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:45.496233 31883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:45.502252 31888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:45.503144 31891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:45.506155 31889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:45.507253 31883 server_base.cc:1034] running on GCE node
I20260928 14:38:45.507503 31883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:45.507800 31883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:45.511068 31883 hybrid_clock.cc:648] HybridClock initialized: now 1790606325511028 us; error 59 us; skew 500 ppm
I20260928 14:38:45.512786 31883 webserver.cc:460] Webserver started at http://127.22.13.65:43639/ using document root <none> and password file <none>
I20260928 14:38:45.513114 31883 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:45.513244 31883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:45.513424 31883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:45.515058 31883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "ba2847687bb44244ad28a14c584aec79"
format_stamp: "Formatted at 2026-09-28 14:38:45 on dist-test-slave-wwwb"
server_key: "fa0b9c384bef2e74e77b3cd601210206"
server_key_iv: "1127a2ae881baa39ccc046f67cb4f2d7"
server_key_version: "encryptionkey@0"
I20260928 14:38:45.515532 31883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "ba2847687bb44244ad28a14c584aec79"
format_stamp: "Formatted at 2026-09-28 14:38:45 on dist-test-slave-wwwb"
server_key: "fa0b9c384bef2e74e77b3cd601210206"
server_key_iv: "1127a2ae881baa39ccc046f67cb4f2d7"
server_key_version: "encryptionkey@0"
I20260928 14:38:45.517735 31883 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260928 14:38:45.518693 31897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:45.518873 31883 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:45.518955 31883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "ba2847687bb44244ad28a14c584aec79"
format_stamp: "Formatted at 2026-09-28 14:38:45 on dist-test-slave-wwwb"
server_key: "fa0b9c384bef2e74e77b3cd601210206"
server_key_iv: "1127a2ae881baa39ccc046f67cb4f2d7"
server_key_version: "encryptionkey@0"
I20260928 14:38:45.519011 31883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:45.530483 31883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:45.530756 31883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:45.530858 31883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:45.531045 31883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:45.531379 31883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:45.531414 31883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:45.531469 31883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:45.531497 31883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:45.537352 31883 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:44257
I20260928 14:38:45.537461 32010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:44257 every 8 connection(s)
I20260928 14:38:45.537827 31883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:38:45.543123 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:45.538703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:51719 (local address 127.22.13.126:46027)
0928 14:38:45.538771 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:45.538773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:45.539056 (+ 283us) server_negotiation.cc:407] Connection header received
0928 14:38:45.539554 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:45.539555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:45.539566 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:45.539620 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:45.540256 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.540739 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.541822 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.542083 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.542138 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:45.542890 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:45.542891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:45.542892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:45.542896 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:45.542909 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:45.542913 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:45.542953 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:45.543019 (+ 66us) server_negotiation.cc:299] Negotiation successful
0928 14:38:45.543041 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:38:45.543315 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:45.538710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:46027 (local address 127.22.13.65:51719)
0928 14:38:45.538930 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:45.538967 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:45.539470 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:45.539686 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:45.539689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:45.539698 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:45.540182 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:45.540187 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.540822 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.540823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:45.541747 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:45.541749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.542155 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.542157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:45.542272 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:45.542757 (+ 485us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:38:45.542757 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:45.542784 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:38:45.542807 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:45.542953 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:45.542956 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:45.542958 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:38:45.543005 (+ 47us) client_negotiation.cc:240] Negotiation successful
0928 14:38:45.543106 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
I20260928 14:38:45.543854 32011 heartbeater.cc:347] Connected to a master server at 127.22.13.126:46027
I20260928 14:38:45.543969 32011 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:45.544174 32011 heartbeater.cc:511] Master 127.22.13.126:46027 requested a full tablet report, sending...
I20260928 14:38:45.544667 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 31883
I20260928 14:38:45.544767 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:38:45.544765 31823 ts_manager.cc:194] Registered new tserver with Master: ba2847687bb44244ad28a14c584aec79 (127.22.13.65:44257)
I20260928 14:38:45.545002 22581 external_mini_cluster.cc:1428] Setting key d021b61261c5045ecd5116fc2b0b282c
I20260928 14:38:45.545589 31823 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.13.65:51719
I20260928 14:38:45.546129 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46027
--builtin_ntp_servers=127.22.13.84:36991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260928 14:38:45.628479 32014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:45.628690 32014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:45.628713 32014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:45.629823 32014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:45.629866 32014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:45.629899 32014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:38:45.631367 32014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:45.631589 32014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:45.631852 32014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:45.636942 32022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:45.636942 32020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:45.636970 32019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:45.637210 32014 server_base.cc:1034] running on GCE node
I20260928 14:38:45.637418 32014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:45.637708 32014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:45.638863 32014 hybrid_clock.cc:648] HybridClock initialized: now 1790606325638850 us; error 29 us; skew 500 ppm
I20260928 14:38:45.640154 32014 webserver.cc:460] Webserver started at http://127.22.13.66:41567/ using document root <none> and password file <none>
I20260928 14:38:45.640445 32014 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:45.640530 32014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:45.640712 32014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:45.641814 32014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "c54f9cb8af0340d8afcb41f46c0ebf4f"
format_stamp: "Formatted at 2026-09-28 14:38:45 on dist-test-slave-wwwb"
server_key: "6da3dee80b5ff5772791796a86153c20"
server_key_iv: "c0fafe0784386d30a581acf9f424172e"
server_key_version: "encryptionkey@0"
I20260928 14:38:45.642246 32014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "c54f9cb8af0340d8afcb41f46c0ebf4f"
format_stamp: "Formatted at 2026-09-28 14:38:45 on dist-test-slave-wwwb"
server_key: "6da3dee80b5ff5772791796a86153c20"
server_key_iv: "c0fafe0784386d30a581acf9f424172e"
server_key_version: "encryptionkey@0"
I20260928 14:38:45.643620 32014 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:45.644351 32028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:45.644520 32014 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:45.644625 32014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "c54f9cb8af0340d8afcb41f46c0ebf4f"
format_stamp: "Formatted at 2026-09-28 14:38:45 on dist-test-slave-wwwb"
server_key: "6da3dee80b5ff5772791796a86153c20"
server_key_iv: "c0fafe0784386d30a581acf9f424172e"
server_key_version: "encryptionkey@0"
I20260928 14:38:45.644703 32014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:45.660357 32014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:45.660719 32014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:45.660879 32014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:45.661109 32014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:45.661459 32014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:45.661516 32014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:45.661557 32014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:45.661585 32014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:45.667405 32014 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:36517
I20260928 14:38:45.667447 32141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:36517 every 8 connection(s)
I20260928 14:38:45.667784 32014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:38:45.670948 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 32014
I20260928 14:38:45.671060 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:38:45.671262 22581 external_mini_cluster.cc:1428] Setting key 4789f4c22175df5d0dbb5340ac3f160a
I20260928 14:38:45.672299 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46027
--builtin_ntp_servers=127.22.13.84:36991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260928 14:38:45.673698 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:45.668506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:44009 (local address 127.22.13.126:46027)
0928 14:38:45.668577 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:45.668579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:45.668834 (+ 255us) server_negotiation.cc:407] Connection header received
0928 14:38:45.669395 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:45.669397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:45.669409 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:45.669467 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:45.670276 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.670803 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.672018 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.672318 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.672398 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:45.673402 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:45.673403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:45.673404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:45.673409 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:45.673426 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:45.673431 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:45.673488 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:45.673557 (+ 69us) server_negotiation.cc:299] Negotiation successful
0928 14:38:45.673587 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 14:38:45.674005 32144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:45.668493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:46027 (local address 127.22.13.66:44009)
0928 14:38:45.668707 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:45.668742 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:45.669284 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:45.669557 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:45.669561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:45.669569 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:45.670164 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:45.670170 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.670902 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.670903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:45.671930 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:45.671933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.672416 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.672417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:45.672615 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:45.673229 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:38:45.673230 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:45.673261 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:38:45.673288 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:45.673494 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:45.673498 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:45.673500 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:38:45.673585 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 14:38:45.673711 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":95,"threads_started":1}
I20260928 14:38:45.674615 32142 heartbeater.cc:347] Connected to a master server at 127.22.13.126:46027
I20260928 14:38:45.674718 32142 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:45.674921 32142 heartbeater.cc:511] Master 127.22.13.126:46027 requested a full tablet report, sending...
I20260928 14:38:45.675269 31823 ts_manager.cc:194] Registered new tserver with Master: c54f9cb8af0340d8afcb41f46c0ebf4f (127.22.13.66:36517)
I20260928 14:38:45.675681 31823 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.13.66:44009
W20260928 14:38:45.755038 32145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:45.755252 32145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:45.755275 32145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:45.756587 32145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:45.756647 32145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:45.756702 32145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:38:45.758172 32145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:45.758430 32145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:45.758740 32145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:45.763737 32151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:45.763713 32153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:45.763993 32145 server_base.cc:1034] running on GCE node
W20260928 14:38:45.763716 32150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:45.764592 32145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:45.764899 32145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:45.766008 32145 hybrid_clock.cc:648] HybridClock initialized: now 1790606325765989 us; error 40 us; skew 500 ppm
I20260928 14:38:45.767442 32145 webserver.cc:460] Webserver started at http://127.22.13.67:43873/ using document root <none> and password file <none>
I20260928 14:38:45.767707 32145 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:45.767757 32145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:45.767931 32145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:45.769363 32145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "88b1bf079e2e436c95ad7c8e6a401054"
format_stamp: "Formatted at 2026-09-28 14:38:45 on dist-test-slave-wwwb"
server_key: "f5749e6bea9da36f60c72e876dc5d4aa"
server_key_iv: "82809eed92e2298ed6455865b1441f76"
server_key_version: "encryptionkey@0"
I20260928 14:38:45.769783 32145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "88b1bf079e2e436c95ad7c8e6a401054"
format_stamp: "Formatted at 2026-09-28 14:38:45 on dist-test-slave-wwwb"
server_key: "f5749e6bea9da36f60c72e876dc5d4aa"
server_key_iv: "82809eed92e2298ed6455865b1441f76"
server_key_version: "encryptionkey@0"
I20260928 14:38:45.771194 32145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:45.771947 32159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:45.772125 32145 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:38:45.772225 32145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "88b1bf079e2e436c95ad7c8e6a401054"
format_stamp: "Formatted at 2026-09-28 14:38:45 on dist-test-slave-wwwb"
server_key: "f5749e6bea9da36f60c72e876dc5d4aa"
server_key_iv: "82809eed92e2298ed6455865b1441f76"
server_key_version: "encryptionkey@0"
I20260928 14:38:45.772310 32145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:45.784490 32145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:45.784895 32145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:45.785063 32145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:45.785298 32145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:45.785653 32145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:45.785708 32145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:45.785751 32145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:45.785779 32145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:45.791582 32145 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:37027
I20260928 14:38:45.791656 32272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:37027 every 8 connection(s)
I20260928 14:38:45.792006 32145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:38:45.797269 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:45.792796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:39405 (local address 127.22.13.126:46027)
0928 14:38:45.792854 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:45.792856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:45.793078 (+ 222us) server_negotiation.cc:407] Connection header received
0928 14:38:45.793559 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:45.793560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:45.793568 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:45.793629 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:45.794357 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.794853 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.795819 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.795999 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.796101 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:45.797051 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:45.797052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:45.797052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:45.797056 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:45.797070 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:45.797074 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:45.797118 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:45.797186 (+ 68us) server_negotiation.cc:299] Negotiation successful
0928 14:38:45.797209 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 14:38:45.797490 32275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:45.792772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:46027 (local address 127.22.13.67:39405)
0928 14:38:45.792948 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:45.792991 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:45.793477 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:45.793685 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:45.793688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:45.793697 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:45.794261 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:45.794266 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.794943 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.794943 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:45.795738 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:45.795739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:45.796140 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:45.796141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:45.796320 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:45.796910 (+ 590us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:38:45.796910 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:45.796940 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:38:45.796963 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:45.797118 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:45.797120 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:45.797122 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:38:45.797169 (+ 47us) client_negotiation.cc:240] Negotiation successful
0928 14:38:45.797247 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
I20260928 14:38:45.797971 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 32145
I20260928 14:38:45.798082 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:38:45.798092 32273 heartbeater.cc:347] Connected to a master server at 127.22.13.126:46027
I20260928 14:38:45.798177 32273 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:45.798288 22581 external_mini_cluster.cc:1428] Setting key df5eb441c0b789454aed04ad47effe80
I20260928 14:38:45.798380 32273 heartbeater.cc:511] Master 127.22.13.126:46027 requested a full tablet report, sending...
I20260928 14:38:45.798795 31823 ts_manager.cc:194] Registered new tserver with Master: 88b1bf079e2e436c95ad7c8e6a401054 (127.22.13.67:37027)
I20260928 14:38:45.799180 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:38:45.799185 31823 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.13.67:39405
W20260928 14:38:45.801123 31862 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:45.800676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55138 (local address 127.22.13.126:46027)
0928 14:38:45.800718 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:45.800720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:45.800830 (+ 110us) server_negotiation.cc:407] Connection header received
0928 14:38:45.800881 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:45.800882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:45.800889 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:45.800933 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:45.801063 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55138: BlockingRecv error: recv got EOF from 127.0.0.1:55138 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 14:38:45.801175 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 31883
I20260928 14:38:45.805711 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 32014
I20260928 14:38:45.810251 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 32145
I20260928 14:38:45.814607 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 31792
2026-09-28T14:38:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (583 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-28T14:38:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:45Z Disabled control of system clock
I20260928 14:38:45.841800 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:39883
--webserver_interface=127.22.13.126
--webserver_port=0
--builtin_ntp_servers=127.22.13.84:35493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:39883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 14:38:45.938657 32284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:45.938877 32284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:45.938908 32284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:45.940065 32284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:45.940105 32284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:45.940119 32284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:45.940131 32284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:45.940143 32284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:45.941494 32284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:35493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:39883
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:39883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:45.941717 32284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:45.942059 32284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:45.946934 32289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:45.946971 32290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:45.946939 32292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:45.947161 32284 server_base.cc:1034] running on GCE node
I20260928 14:38:45.947439 32284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:45.947751 32284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:45.948895 32284 hybrid_clock.cc:648] HybridClock initialized: now 1790606325948883 us; error 26 us; skew 500 ppm
I20260928 14:38:45.950228 32284 webserver.cc:460] Webserver started at http://127.22.13.126:38665/ using document root <none> and password file <none>
I20260928 14:38:45.950480 32284 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:45.950529 32284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:45.950699 32284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:45.951763 32284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "6d7ba013fb034160ae1c095a7d53a93e"
format_stamp: "Formatted at 2026-09-28 14:38:45 on dist-test-slave-wwwb"
server_key: "228cf53f9d5e16b91ce7470be4e2f002"
server_key_iv: "521713b65aed69e231870c405429e91b"
server_key_version: "encryptionkey@0"
I20260928 14:38:45.952135 32284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "6d7ba013fb034160ae1c095a7d53a93e"
format_stamp: "Formatted at 2026-09-28 14:38:45 on dist-test-slave-wwwb"
server_key: "228cf53f9d5e16b91ce7470be4e2f002"
server_key_iv: "521713b65aed69e231870c405429e91b"
server_key_version: "encryptionkey@0"
I20260928 14:38:45.953411 32284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:45.954141 32298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:45.954394 32284 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:38:45.954480 32284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "6d7ba013fb034160ae1c095a7d53a93e"
format_stamp: "Formatted at 2026-09-28 14:38:45 on dist-test-slave-wwwb"
server_key: "228cf53f9d5e16b91ce7470be4e2f002"
server_key_iv: "521713b65aed69e231870c405429e91b"
server_key_version: "encryptionkey@0"
I20260928 14:38:45.954536 32284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:45.957893 32284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:45.958251 32284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:45.958412 32284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:45.962388 32284 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:39883
I20260928 14:38:45.962414 32350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:39883 every 8 connection(s)
I20260928 14:38:45.962786 32284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:45.963424 32351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:45.966171 32351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e: Bootstrap starting.
I20260928 14:38:45.966446 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 32284
I20260928 14:38:45.966562 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:45.966755 22581 external_mini_cluster.cc:1428] Setting key 08a6df15b7743c9336cd6d21cec8da28
I20260928 14:38:45.966922 32351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:45.967245 32351 log.cc:824] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:45.968010 32351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e: No bootstrap required, opened a new log
I20260928 14:38:45.969350 32354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:45.967301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59952 (local address 127.22.13.126:39883)
0928 14:38:45.967480 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:45.967495 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:45.967549 (+ 54us) server_negotiation.cc:407] Connection header received
0928 14:38:45.968042 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:45.968049 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:45.968112 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:45.968263 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:45.968901 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:45.968904 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:45.968904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:45.968908 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:45.968947 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:45.968957 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:45.969022 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:45.969095 (+ 73us) server_negotiation.cc:299] Negotiation successful
0928 14:38:45.969167 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
I20260928 14:38:45.969911 32351 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7ba013fb034160ae1c095a7d53a93e" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39883 } }
I20260928 14:38:45.970120 32351 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:45.970166 32351 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7ba013fb034160ae1c095a7d53a93e, State: Initialized, Role: FOLLOWER
I20260928 14:38:45.970284 32351 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7ba013fb034160ae1c095a7d53a93e" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39883 } }
I20260928 14:38:45.970359 32351 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:45.970407 32351 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:45.970453 32351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:45.971256 32351 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7ba013fb034160ae1c095a7d53a93e" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39883 } }
I20260928 14:38:45.971381 32351 leader_election.cc:302] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d7ba013fb034160ae1c095a7d53a93e; no voters:
I20260928 14:38:45.971542 32351 leader_election.cc:288] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:45.971565 32356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:45.971714 32356 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [term 1 LEADER]: Becoming Leader. State: Replica: 6d7ba013fb034160ae1c095a7d53a93e, State: Running, Role: LEADER
I20260928 14:38:45.971820 32351 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:45.971925 32356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7ba013fb034160ae1c095a7d53a93e" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39883 } }
I20260928 14:38:45.972512 32357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d7ba013fb034160ae1c095a7d53a93e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7ba013fb034160ae1c095a7d53a93e" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39883 } } }
I20260928 14:38:45.972518 32358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d7ba013fb034160ae1c095a7d53a93e. Latest consensus state: current_term: 1 leader_uuid: "6d7ba013fb034160ae1c095a7d53a93e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d7ba013fb034160ae1c095a7d53a93e" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39883 } } }
I20260928 14:38:45.972613 32357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:45.972635 32358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:45.972815 32364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:45.973395 32364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:45.974979 32364 catalog_manager.cc:1347] Generated new cluster ID: d19e49dbbab045ae9f98b0bc72450101
I20260928 14:38:45.975032 32364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:38:45.981611 32364 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:45.981796 32364 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:45.986946 32364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d7ba013fb034160ae1c095a7d53a93e: Generated new TSK 0
I20260928 14:38:45.987165 32364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:38:45.994663 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:39883
--builtin_ntp_servers=127.22.13.84:35493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 14:38:46.079077 32375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:46.079327 32375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:46.079366 32375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:46.080667 32375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:46.080732 32375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:46.080778 32375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:38:46.082284 32375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:35493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:39883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:46.082579 32375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:46.082886 32375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:46.088824 32383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:46.088824 32381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:46.089051 32375 server_base.cc:1034] running on GCE node
W20260928 14:38:46.088868 32380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:46.089324 32375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:46.089602 32375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:46.090767 32375 hybrid_clock.cc:648] HybridClock initialized: now 1790606326090752 us; error 30 us; skew 500 ppm
I20260928 14:38:46.092141 32375 webserver.cc:460] Webserver started at http://127.22.13.65:33673/ using document root <none> and password file <none>
I20260928 14:38:46.092407 32375 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:46.092481 32375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:46.092636 32375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:46.093750 32375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "1e11398dc3ff403c92435a3e3abb702c"
format_stamp: "Formatted at 2026-09-28 14:38:46 on dist-test-slave-wwwb"
server_key: "c92dd0cf806d123d49b4d3bb9bebac5f"
server_key_iv: "ff00d2961f45c444a1376aa93db0b522"
server_key_version: "encryptionkey@0"
I20260928 14:38:46.094192 32375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "1e11398dc3ff403c92435a3e3abb702c"
format_stamp: "Formatted at 2026-09-28 14:38:46 on dist-test-slave-wwwb"
server_key: "c92dd0cf806d123d49b4d3bb9bebac5f"
server_key_iv: "ff00d2961f45c444a1376aa93db0b522"
server_key_version: "encryptionkey@0"
I20260928 14:38:46.095651 32375 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:46.096499 32389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:46.096700 32375 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:38:46.096789 32375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "1e11398dc3ff403c92435a3e3abb702c"
format_stamp: "Formatted at 2026-09-28 14:38:46 on dist-test-slave-wwwb"
server_key: "c92dd0cf806d123d49b4d3bb9bebac5f"
server_key_iv: "ff00d2961f45c444a1376aa93db0b522"
server_key_version: "encryptionkey@0"
I20260928 14:38:46.096885 32375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:46.100459 32375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:46.100788 32375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:46.100947 32375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:46.101194 32375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:46.101527 32375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:46.101583 32375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:46.101624 32375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:46.101651 32375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:46.108157 32375 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:39469
I20260928 14:38:46.108230 32502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:39469 every 8 connection(s)
I20260928 14:38:46.108615 32375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:38:46.110918 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 32375
I20260928 14:38:46.111033 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:38:46.111249 22581 external_mini_cluster.cc:1428] Setting key e307fae5aa473817639ef991b1c18675
I20260928 14:38:46.112154 32354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:46.109440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:55393 (local address 127.22.13.126:39883)
0928 14:38:46.109495 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:46.109497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:46.109751 (+ 254us) server_negotiation.cc:407] Connection header received
0928 14:38:46.110302 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:46.110303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:46.110309 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:46.110386 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:46.111844 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:46.111846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:46.111846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:46.111854 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:46.111876 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:46.111880 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:46.111977 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:46.112078 (+ 101us) server_negotiation.cc:299] Negotiation successful
0928 14:38:46.112101 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:38:46.112357 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:39883
--builtin_ntp_servers=127.22.13.84:35493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260928 14:38:46.112462 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:46.109405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39883 (local address 127.22.13.65:55393)
0928 14:38:46.109604 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:46.109647 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:46.110198 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:46.110452 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:46.110456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:46.110464 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:46.111668 (+ 1204us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:38:46.111669 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:46.111703 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:38:46.111736 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:46.111975 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:46.111981 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:46.111985 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:38:46.112075 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 14:38:46.112149 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260928 14:38:46.112943 32503 heartbeater.cc:347] Connected to a master server at 127.22.13.126:39883
I20260928 14:38:46.113067 32503 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:46.113236 32503 heartbeater.cc:511] Master 127.22.13.126:39883 requested a full tablet report, sending...
I20260928 14:38:46.113821 32315 ts_manager.cc:194] Registered new tserver with Master: 1e11398dc3ff403c92435a3e3abb702c (127.22.13.65:39469)
W20260928 14:38:46.194818 32506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:46.195086 32506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:46.195128 32506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:46.196349 32506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:46.196436 32506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:46.196483 32506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:38:46.197984 32506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:35493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:39883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:46.198282 32506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:46.198592 32506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:46.203765 32512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:46.203737 32511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:46.204026 32514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:46.203969 32506 server_base.cc:1034] running on GCE node
I20260928 14:38:46.204296 32506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:46.204574 32506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:46.205746 32506 hybrid_clock.cc:648] HybridClock initialized: now 1790606326205731 us; error 26 us; skew 500 ppm
I20260928 14:38:46.207276 32506 webserver.cc:460] Webserver started at http://127.22.13.66:35579/ using document root <none> and password file <none>
I20260928 14:38:46.207530 32506 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:46.207599 32506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:46.207738 32506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:46.208820 32506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "3bb69bd28e464ecf9c9ace87307d3168"
format_stamp: "Formatted at 2026-09-28 14:38:46 on dist-test-slave-wwwb"
server_key: "b8ced3e7bed5939356ea2b07aa759a82"
server_key_iv: "37913ddf65ca9378294ea0260b967675"
server_key_version: "encryptionkey@0"
I20260928 14:38:46.209223 32506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "3bb69bd28e464ecf9c9ace87307d3168"
format_stamp: "Formatted at 2026-09-28 14:38:46 on dist-test-slave-wwwb"
server_key: "b8ced3e7bed5939356ea2b07aa759a82"
server_key_iv: "37913ddf65ca9378294ea0260b967675"
server_key_version: "encryptionkey@0"
I20260928 14:38:46.210651 32506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:38:46.211383 32520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:46.211553 32506 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:46.211652 32506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "3bb69bd28e464ecf9c9ace87307d3168"
format_stamp: "Formatted at 2026-09-28 14:38:46 on dist-test-slave-wwwb"
server_key: "b8ced3e7bed5939356ea2b07aa759a82"
server_key_iv: "37913ddf65ca9378294ea0260b967675"
server_key_version: "encryptionkey@0"
I20260928 14:38:46.211735 32506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:46.215188 32506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:46.215466 32506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:46.215608 32506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:46.215816 32506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:46.216125 32506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:46.216178 32506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:46.216218 32506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:46.216245 32506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:46.222370 32506 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:43775
I20260928 14:38:46.222424 32633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:43775 every 8 connection(s)
I20260928 14:38:46.222755 32506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:38:46.225620 32354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:46.223477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:33021 (local address 127.22.13.126:39883)
0928 14:38:46.223545 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:46.223547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:46.223796 (+ 249us) server_negotiation.cc:407] Connection header received
0928 14:38:46.224241 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:46.224242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:46.224247 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:46.224313 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:46.225387 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:46.225389 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:46.225389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:46.225393 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:46.225407 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:46.225411 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:46.225465 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:46.225552 (+ 87us) server_negotiation.cc:299] Negotiation successful
0928 14:38:46.225566 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:38:46.225831 32636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:46.223484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39883 (local address 127.22.13.66:33021)
0928 14:38:46.223661 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:46.223702 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:46.224158 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:46.224369 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:46.224373 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:46.224382 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:46.225231 (+ 849us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:38:46.225231 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:46.225262 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:38:46.225289 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:46.225465 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:46.225468 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:46.225470 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:38:46.225542 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 14:38:46.225592 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260928 14:38:46.226337 32634 heartbeater.cc:347] Connected to a master server at 127.22.13.126:39883
I20260928 14:38:46.226441 32634 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:46.226563 32634 heartbeater.cc:511] Master 127.22.13.126:39883 requested a full tablet report, sending...
I20260928 14:38:46.226915 32315 ts_manager.cc:194] Registered new tserver with Master: 3bb69bd28e464ecf9c9ace87307d3168 (127.22.13.66:43775)
I20260928 14:38:46.227430 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 32506
I20260928 14:38:46.227548 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:38:46.227742 22581 external_mini_cluster.cc:1428] Setting key 92e4f9cd94ffb9b97cc0012d805fb0a8
I20260928 14:38:46.228550 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:39883
--builtin_ntp_servers=127.22.13.84:35493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 14:38:46.308772 32637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:46.309013 32637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:46.309036 32637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:46.310280 32637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:46.310323 32637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:46.310349 32637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:38:46.311657 32637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:35493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:39883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:46.311889 32637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:46.312194 32637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:46.317348 32645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:46.317462 32643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:46.317341 32642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:46.317571 32637 server_base.cc:1034] running on GCE node
I20260928 14:38:46.317785 32637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:46.318118 32637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:46.319298 32637 hybrid_clock.cc:648] HybridClock initialized: now 1790606326319284 us; error 26 us; skew 500 ppm
I20260928 14:38:46.320735 32637 webserver.cc:460] Webserver started at http://127.22.13.67:34793/ using document root <none> and password file <none>
I20260928 14:38:46.321005 32637 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:46.321080 32637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:46.321233 32637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:46.322474 32637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "0df0d821fce74bfdac90db056da48feb"
format_stamp: "Formatted at 2026-09-28 14:38:46 on dist-test-slave-wwwb"
server_key: "49577cb563e7454a309c1643cb15c0e7"
server_key_iv: "7bc2d5845c7f4c820ec632d0ae9286f2"
server_key_version: "encryptionkey@0"
I20260928 14:38:46.322842 32637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "0df0d821fce74bfdac90db056da48feb"
format_stamp: "Formatted at 2026-09-28 14:38:46 on dist-test-slave-wwwb"
server_key: "49577cb563e7454a309c1643cb15c0e7"
server_key_iv: "7bc2d5845c7f4c820ec632d0ae9286f2"
server_key_version: "encryptionkey@0"
I20260928 14:38:46.324198 32637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:38:46.325004 32651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:46.325172 32637 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:46.325274 32637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "0df0d821fce74bfdac90db056da48feb"
format_stamp: "Formatted at 2026-09-28 14:38:46 on dist-test-slave-wwwb"
server_key: "49577cb563e7454a309c1643cb15c0e7"
server_key_iv: "7bc2d5845c7f4c820ec632d0ae9286f2"
server_key_version: "encryptionkey@0"
I20260928 14:38:46.325361 32637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:46.328860 32637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:46.329144 32637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:46.329281 32637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:46.329504 32637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:46.329813 32637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:46.329864 32637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:46.329895 32637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:46.329916 32637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:46.335651 32637 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:46419
I20260928 14:38:46.335727 32764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:46419 every 8 connection(s)
I20260928 14:38:46.336078 32637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:38:46.339003 32354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:46.336869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:49677 (local address 127.22.13.126:39883)
0928 14:38:46.336961 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:46.336964 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:46.337169 (+ 205us) server_negotiation.cc:407] Connection header received
0928 14:38:46.337620 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:46.337621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:46.337626 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:46.337679 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:46.338812 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:46.338813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:46.338813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:38:46.338817 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:46.338832 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:38:46.338837 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:46.338887 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:46.338936 (+ 49us) server_negotiation.cc:299] Negotiation successful
0928 14:38:46.338948 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 14:38:46.339224 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:46.336846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39883 (local address 127.22.13.67:49677)
0928 14:38:46.337051 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:46.337085 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:46.337540 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:46.337732 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:46.337735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:46.337744 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:46.338638 (+ 894us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:38:46.338639 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:46.338670 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:38:46.338718 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:46.338885 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:46.338887 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:46.338890 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:38:46.338944 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 14:38:46.338994 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":113,"threads_started":1}
I20260928 14:38:46.339797 32765 heartbeater.cc:347] Connected to a master server at 127.22.13.126:39883
I20260928 14:38:46.339875 32765 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:46.339991 32765 heartbeater.cc:511] Master 127.22.13.126:39883 requested a full tablet report, sending...
I20260928 14:38:46.340319 32315 ts_manager.cc:194] Registered new tserver with Master: 0df0d821fce74bfdac90db056da48feb (127.22.13.67:46419)
I20260928 14:38:46.343523 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 32637
I20260928 14:38:46.343640 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:38:46.343842 22581 external_mini_cluster.cc:1428] Setting key 637d569f49cd6f601ab63c69e13feacd
I20260928 14:38:46.344718 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 14:38:46.346642 32354 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:38:46.346121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59958 (local address 127.22.13.126:39883)
0928 14:38:46.346166 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:46.346168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:46.346288 (+ 120us) server_negotiation.cc:407] Connection header received
0928 14:38:46.346330 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:46.346330 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:46.346335 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:46.346426 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:46.346568 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59958: BlockingRecv error: recv got EOF from 127.0.0.1:59958 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:38:46.346680 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 32375
I20260928 14:38:46.351107 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 32506
I20260928 14:38:46.355666 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 32637
I20260928 14:38:46.360114 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 32284
2026-09-28T14:38:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (545 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:38:46 dist-test-slave-wwwb krb5kdc[307](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:38:46 dist-test-slave-wwwb krb5kdc[307](info): set up 2 sockets
Sep 28 14:38:46 dist-test-slave-wwwb krb5kdc[307](info): commencing operation
krb5kdc: starting...
W20260928 14:38:47.391018 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:38:47 dist-test-slave-wwwb krb5kdc[307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:38:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:38:47.508662 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:36381
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:42913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:36381
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:47.590474 320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:47.590694 320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:47.590718 320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:47.592288 320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:38:47.592352 320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:47.592383 320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:47.592407 320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:38:47.592478 320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:38:47.594507 320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:42913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:36381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:36381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:47.594790 320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:47.595067 320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:47.600056 326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:47.600061 325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:47.600126 328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:47.600310 320 server_base.cc:1034] running on GCE node
I20260928 14:38:47.600584 320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:47.600934 320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:47.602099 320 hybrid_clock.cc:648] HybridClock initialized: now 1790606327602078 us; error 33 us; skew 500 ppm
Sep 28 14:38:47 dist-test-slave-wwwb krb5kdc[307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606327, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:47.604629 320 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:38:47.605086 320 webserver.cc:460] Webserver started at http://127.22.13.126:35311/ using document root <none> and password file <none>
I20260928 14:38:47.605319 320 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:47.605396 320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:47.605556 320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:47.606726 320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "843650b315ac4bb0a708b8e4eb16f0c6"
format_stamp: "Formatted at 2026-09-28 14:38:47 on dist-test-slave-wwwb"
server_key: "c2f789708454915a2825ce8ad8c946ce"
server_key_iv: "6adc033cb5cee5eee73a7f97b7489f40"
server_key_version: "encryptionkey@0"
I20260928 14:38:47.607105 320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "843650b315ac4bb0a708b8e4eb16f0c6"
format_stamp: "Formatted at 2026-09-28 14:38:47 on dist-test-slave-wwwb"
server_key: "c2f789708454915a2825ce8ad8c946ce"
server_key_iv: "6adc033cb5cee5eee73a7f97b7489f40"
server_key_version: "encryptionkey@0"
I20260928 14:38:47.608426 320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:38:47.609138 335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:47.609308 320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:47.609423 320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "843650b315ac4bb0a708b8e4eb16f0c6"
format_stamp: "Formatted at 2026-09-28 14:38:47 on dist-test-slave-wwwb"
server_key: "c2f789708454915a2825ce8ad8c946ce"
server_key_iv: "6adc033cb5cee5eee73a7f97b7489f40"
server_key_version: "encryptionkey@0"
I20260928 14:38:47.609500 320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:47.628060 320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:47.630048 320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:47.630219 320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:47.634084 320 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:36381
I20260928 14:38:47.634122 387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:36381 every 8 connection(s)
I20260928 14:38:47.634501 320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:38:47.635535 388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:47.637837 388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6: Bootstrap starting.
I20260928 14:38:47.638468 388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:47.638767 388 log.cc:824] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:47.639477 388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6: No bootstrap required, opened a new log
I20260928 14:38:47.641108 388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843650b315ac4bb0a708b8e4eb16f0c6" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36381 } }
I20260928 14:38:47.641249 388 raft_consensus.cc:374] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:47.641297 388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843650b315ac4bb0a708b8e4eb16f0c6, State: Initialized, Role: FOLLOWER
I20260928 14:38:47.641462 388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843650b315ac4bb0a708b8e4eb16f0c6" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36381 } }
I20260928 14:38:47.641525 388 raft_consensus.cc:388] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:38:47.641546 388 raft_consensus.cc:482] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:38:47.641615 388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:47.642588 388 raft_consensus.cc:504] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843650b315ac4bb0a708b8e4eb16f0c6" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36381 } }
I20260928 14:38:47.642706 388 leader_election.cc:302] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 843650b315ac4bb0a708b8e4eb16f0c6; no voters:
I20260928 14:38:47.642907 388 leader_election.cc:288] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:38:47.642966 391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:47.643131 391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [term 1 LEADER]: Becoming Leader. State: Replica: 843650b315ac4bb0a708b8e4eb16f0c6, State: Running, Role: LEADER
I20260928 14:38:47.643198 388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:38:47.643225 391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843650b315ac4bb0a708b8e4eb16f0c6" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36381 } }
I20260928 14:38:47.643564 393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 843650b315ac4bb0a708b8e4eb16f0c6. Latest consensus state: current_term: 1 leader_uuid: "843650b315ac4bb0a708b8e4eb16f0c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843650b315ac4bb0a708b8e4eb16f0c6" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36381 } } }
I20260928 14:38:47.643662 393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:47.643723 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 320
I20260928 14:38:47.643822 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:38:47.644003 399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:38:47.644037 22581 external_mini_cluster.cc:1428] Setting key e8dda35aae7ebb70020fe4a0f2e36ce4
I20260928 14:38:47.645057 399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:38:47.645064 392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "843650b315ac4bb0a708b8e4eb16f0c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843650b315ac4bb0a708b8e4eb16f0c6" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36381 } } }
I20260928 14:38:47.645431 392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6 [sys.catalog]: This master's current role is: LEADER
I20260928 14:38:47.650224 399 catalog_manager.cc:1347] Generated new cluster ID: 9bd18984851941c7a62884f3984147b5
I20260928 14:38:47.650291 399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:38:47 dist-test-slave-wwwb krb5kdc[307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:47.656308 410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:47.644677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60364 (local address 127.22.13.126:36381)
0928 14:38:47.645413 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:47.645523 (+ 110us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:47.645565 (+ 42us) server_negotiation.cc:407] Connection header received
0928 14:38:47.645692 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:47.645698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:47.645777 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:47.645921 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:47.646729 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:47.647608 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:47.648728 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:47.649138 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:47.649214 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:47.652001 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:47.652006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:47.652006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:47.652031 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:47.654578 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:47.654983 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:47.654988 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:47.654989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:47.655126 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:47.655336 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:47.655337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:47.655338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:47.655670 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:47.655838 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:47.655921 (+ 83us) server_negotiation.cc:299] Negotiation successful
0928 14:38:47.656076 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":398,"threads_started":1}
I20260928 14:38:47.671290 399 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:38:47.671852 399 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:38:47.678761 399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6: Generated new TSK 0
I20260928 14:38:47.678965 399 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:38:47.706148 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36381
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:42913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:47.787441 415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:47.787679 415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:47.787703 415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:47.788904 415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:47.788952 415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:47.788980 415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:38:47.790596 415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:42913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:47.790869 415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:47.791109 415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:47.796162 421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:47.796222 420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:47.796167 423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:47.796509 415 server_base.cc:1034] running on GCE node
I20260928 14:38:47.796734 415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:47.797003 415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:47.798162 415 hybrid_clock.cc:648] HybridClock initialized: now 1790606327798156 us; error 33 us; skew 500 ppm
Sep 28 14:38:47 dist-test-slave-wwwb krb5kdc[307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606327, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:47.800604 415 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:38:47.801052 415 webserver.cc:460] Webserver started at http://127.22.13.65:40123/ using document root <none> and password file <none>
I20260928 14:38:47.801285 415 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:47.801353 415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:47.801509 415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:47.802647 415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "a35e17b8fb104927be8236cf7f5b4cde"
format_stamp: "Formatted at 2026-09-28 14:38:47 on dist-test-slave-wwwb"
server_key: "3619b0c67a169b1c01f126d8bd854c55"
server_key_iv: "89b1aa888e152b67e26bf6bffc067d92"
server_key_version: "encryptionkey@0"
I20260928 14:38:47.803027 415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "a35e17b8fb104927be8236cf7f5b4cde"
format_stamp: "Formatted at 2026-09-28 14:38:47 on dist-test-slave-wwwb"
server_key: "3619b0c67a169b1c01f126d8bd854c55"
server_key_iv: "89b1aa888e152b67e26bf6bffc067d92"
server_key_version: "encryptionkey@0"
I20260928 14:38:47.804358 415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:38:47.805101 430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:47.805274 415 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:47.805389 415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "a35e17b8fb104927be8236cf7f5b4cde"
format_stamp: "Formatted at 2026-09-28 14:38:47 on dist-test-slave-wwwb"
server_key: "3619b0c67a169b1c01f126d8bd854c55"
server_key_iv: "89b1aa888e152b67e26bf6bffc067d92"
server_key_version: "encryptionkey@0"
I20260928 14:38:47.805472 415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:47.826352 415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:47.828476 415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:47.828665 415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:47.828924 415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:47.829265 415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:47.829319 415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:47.829360 415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:47.829386 415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:47.835467 415 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:42603
I20260928 14:38:47.835511 544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:42603 every 8 connection(s)
I20260928 14:38:47.835887 415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:38:47.841221 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 415
I20260928 14:38:47.841337 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:38:47.841571 22581 external_mini_cluster.cc:1428] Setting key 1c339aec503cb1362bdb0cf297af667f
Sep 28 14:38:47 dist-test-slave-wwwb krb5kdc[307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606327, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:47.846050 410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:47.836659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:37543 (local address 127.22.13.126:36381)
0928 14:38:47.836733 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:47.836735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:47.836951 (+ 216us) server_negotiation.cc:407] Connection header received
0928 14:38:47.837424 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:47.837425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:47.837438 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:47.837498 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:47.838473 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:47.838976 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:47.839946 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:47.840137 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:47.840188 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:47.842751 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:47.842752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:47.842753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:47.842761 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:47.844306 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:47.844882 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:47.844883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:47.844884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:47.844942 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:47.845246 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:47.845246 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:47.845247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:47.845372 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:47.845425 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:47.845810 (+ 385us) server_negotiation.cc:299] Negotiation successful
0928 14:38:47.845900 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 14:38:47.846313 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:47.836635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36381 (local address 127.22.13.65:37543)
0928 14:38:47.836796 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:47.836852 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:47.837338 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:47.837554 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:47.837557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:47.838164 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:47.838378 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:47.838384 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:47.839068 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:47.839069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:47.839855 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:47.839857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:47.840212 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:47.840212 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:47.840318 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:47.840798 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:47.840806 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:47.842560 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:47.844423 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:47.844432 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:47.844434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:47.844805 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:47.845015 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:47.845016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:47.845017 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:47.845181 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:47.845439 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:47.845445 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:47.845657 (+ 212us) client_negotiation.cc:769] Sending connection context
0928 14:38:47.845787 (+ 130us) client_negotiation.cc:240] Negotiation successful
0928 14:38:47.846028 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260928 14:38:47.846917 545 heartbeater.cc:347] Connected to a master server at 127.22.13.126:36381
I20260928 14:38:47.847031 545 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:47.847254 545 heartbeater.cc:511] Master 127.22.13.126:36381 requested a full tablet report, sending...
I20260928 14:38:47.847803 352 ts_manager.cc:194] Registered new tserver with Master: a35e17b8fb104927be8236cf7f5b4cde (127.22.13.65:42603)
I20260928 14:38:47.848412 352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:37543
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:38:47.866962 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36381
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:42913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:47.947510 550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:47.947763 550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:47.947805 550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:47.949011 550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:47.949091 550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:47.949137 550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:38:47.950644 550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:42913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:47.950932 550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:47.951190 550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:47.956074 558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:47.956097 555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:47.956097 556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:47.956326 550 server_base.cc:1034] running on GCE node
I20260928 14:38:47.956568 550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:47.956856 550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:47.958005 550 hybrid_clock.cc:648] HybridClock initialized: now 1790606327958001 us; error 47 us; skew 500 ppm
Sep 28 14:38:47 dist-test-slave-wwwb krb5kdc[307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606327, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:47.960438 550 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:38:47.960868 550 webserver.cc:460] Webserver started at http://127.22.13.66:37469/ using document root <none> and password file <none>
I20260928 14:38:47.961090 550 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:47.961156 550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:47.961308 550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:47.962464 550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "0b0c5b657e7143aa8ade5cc148f30ad5"
format_stamp: "Formatted at 2026-09-28 14:38:47 on dist-test-slave-wwwb"
server_key: "9f2826e9f6792f7ca4205d748fa08a78"
server_key_iv: "fc3dbf162de5fb04eea7d1be8d295a2c"
server_key_version: "encryptionkey@0"
I20260928 14:38:47.962822 550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "0b0c5b657e7143aa8ade5cc148f30ad5"
format_stamp: "Formatted at 2026-09-28 14:38:47 on dist-test-slave-wwwb"
server_key: "9f2826e9f6792f7ca4205d748fa08a78"
server_key_iv: "fc3dbf162de5fb04eea7d1be8d295a2c"
server_key_version: "encryptionkey@0"
I20260928 14:38:47.964149 550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:38:47.964840 565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:47.965016 550 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:38:47.965112 550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "0b0c5b657e7143aa8ade5cc148f30ad5"
format_stamp: "Formatted at 2026-09-28 14:38:47 on dist-test-slave-wwwb"
server_key: "9f2826e9f6792f7ca4205d748fa08a78"
server_key_iv: "fc3dbf162de5fb04eea7d1be8d295a2c"
server_key_version: "encryptionkey@0"
I20260928 14:38:47.965193 550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:47.980329 550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:47.982353 550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:47.982527 550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:47.982771 550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:47.983091 550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:47.983146 550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:47.983180 550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:47.983207 550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:47.988993 550 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:41039
I20260928 14:38:47.989039 680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:41039 every 8 connection(s)
I20260928 14:38:47.989389 550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:38:47.992053 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 550
I20260928 14:38:47.992161 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:38:47.992377 22581 external_mini_cluster.cc:1428] Setting key b5020cc3dc5305568e0a775ea58aa052
Sep 28 14:38:47 dist-test-slave-wwwb krb5kdc[307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606327, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:48.000075 410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:47.990223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:42763 (local address 127.22.13.126:36381)
0928 14:38:47.990305 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:47.990308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:47.990496 (+ 188us) server_negotiation.cc:407] Connection header received
0928 14:38:47.990966 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:47.990966 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:47.990976 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:47.991024 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:47.992049 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:47.992712 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:47.994241 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:47.994435 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:47.994548 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:47.996999 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:47.997000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:47.997001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:47.997008 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:47.998562 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:47.999053 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:47.999055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:47.999055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:47.999119 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:47.999400 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:47.999402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:47.999402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:47.999527 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:47.999606 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:47.999897 (+ 291us) server_negotiation.cc:299] Negotiation successful
0928 14:38:47.999985 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 14:38:48.000290 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:47.990179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36381 (local address 127.22.13.66:42763)
0928 14:38:47.990376 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:47.990413 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:47.990888 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:47.991085 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:47.991088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:47.991673 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:47.991946 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:47.991952 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:47.992804 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:47.992805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:47.994160 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:47.994163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:47.994570 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:47.994572 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:47.994734 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:47.995289 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:47.995309 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:47.996899 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:47.998691 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:47.998699 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:47.998702 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:47.998946 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:47.999196 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:47.999196 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:47.999197 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:47.999316 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:47.999597 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:47.999604 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:47.999754 (+ 150us) client_negotiation.cc:769] Sending connection context
0928 14:38:47.999859 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 14:38:48.000026 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
I20260928 14:38:48.000898 681 heartbeater.cc:347] Connected to a master server at 127.22.13.126:36381
I20260928 14:38:48.001012 681 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:48.001194 681 heartbeater.cc:511] Master 127.22.13.126:36381 requested a full tablet report, sending...
I20260928 14:38:48.001664 352 ts_manager.cc:194] Registered new tserver with Master: 0b0c5b657e7143aa8ade5cc148f30ad5 (127.22.13.66:41039)
I20260928 14:38:48.002120 352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:42763
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:38:48.017168 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36381
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:42913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:38:48.097301 686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:38:48.097519 686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:38:48.097543 686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:38:48.098687 686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:38:48.098728 686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:38:48.098755 686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:38:48.100140 686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:42913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:38:48.100329 686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:38:48.100515 686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:38:48.105531 691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:38:48.105599 694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:48.105751 686 server_base.cc:1034] running on GCE node
W20260928 14:38:48.105538 692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:38:48.106053 686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:38:48.106348 686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:38:48.107554 686 hybrid_clock.cc:648] HybridClock initialized: now 1790606328107533 us; error 30 us; skew 500 ppm
Sep 28 14:38:48 dist-test-slave-wwwb krb5kdc[307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606328, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:38:48.110172 686 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:38:48.110641 686 webserver.cc:460] Webserver started at http://127.22.13.67:37391/ using document root <none> and password file <none>
I20260928 14:38:48.110850 686 fs_manager.cc:359] Metadata directory not provided
I20260928 14:38:48.110891 686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:38:48.110982 686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:38:48.112095 686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "1af134aceb0241f0990869c6a801980d"
format_stamp: "Formatted at 2026-09-28 14:38:48 on dist-test-slave-wwwb"
server_key: "609611124e06c55308554121a5952fcc"
server_key_iv: "225f186bf095a19501b5c6cbfb86cbc4"
server_key_version: "encryptionkey@0"
I20260928 14:38:48.112468 686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "1af134aceb0241f0990869c6a801980d"
format_stamp: "Formatted at 2026-09-28 14:38:48 on dist-test-slave-wwwb"
server_key: "609611124e06c55308554121a5952fcc"
server_key_iv: "225f186bf095a19501b5c6cbfb86cbc4"
server_key_version: "encryptionkey@0"
I20260928 14:38:48.113797 686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:38:48.114550 701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:48.114697 686 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:48.114777 686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "1af134aceb0241f0990869c6a801980d"
format_stamp: "Formatted at 2026-09-28 14:38:48 on dist-test-slave-wwwb"
server_key: "609611124e06c55308554121a5952fcc"
server_key_iv: "225f186bf095a19501b5c6cbfb86cbc4"
server_key_version: "encryptionkey@0"
I20260928 14:38:48.114830 686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:38:48.121230 686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:38:48.123440 686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:38:48.123553 686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:38:48.123732 686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:38:48.124076 686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:38:48.124141 686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:48.124190 686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:38:48.124217 686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:38:48.129778 686 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:33075
I20260928 14:38:48.129920 815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:33075 every 8 connection(s)
I20260928 14:38:48.130224 686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:38:48.132042 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 686
I20260928 14:38:48.132184 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:38:48.132382 22581 external_mini_cluster.cc:1428] Setting key 4abc3b38642cef79227f6b0b8fbf05e6
Sep 28 14:38:48 dist-test-slave-wwwb krb5kdc[307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606328, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:38:48.139955 410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:48.131001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:48103 (local address 127.22.13.126:36381)
0928 14:38:48.131061 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:48.131064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:48.131296 (+ 232us) server_negotiation.cc:407] Connection header received
0928 14:38:48.131828 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:48.131829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:48.131841 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:48.131906 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:48.133061 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.133677 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.134756 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.134953 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.135058 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:48.137265 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:48.137266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:48.137266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:48.137274 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:48.138561 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:48.138984 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:48.138986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:48.138986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:48.139045 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:48.139298 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:48.139299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:48.139299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:48.139422 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:48.139494 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:48.139772 (+ 278us) server_negotiation.cc:299] Negotiation successful
0928 14:38:48.139854 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 14:38:48.140136 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:48.130986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36381 (local address 127.22.13.67:48103)
0928 14:38:48.131151 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:48.131200 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:48.131722 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:48.131972 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:48.131976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:48.132710 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:38:48.132976 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:48.132983 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.133788 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.133789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:48.134662 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:48.134665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.135064 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.135065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:48.135219 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:48.135745 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:38:48.135754 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:38:48.137167 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:38:48.138657 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:48.138664 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:48.138666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:48.138891 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:48.139111 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:38:48.139112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:38:48.139112 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:38:48.139228 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:38:48.139497 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:38:48.139504 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:38:48.139644 (+ 140us) client_negotiation.cc:769] Sending connection context
0928 14:38:48.139749 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 14:38:48.139910 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260928 14:38:48.140699 816 heartbeater.cc:347] Connected to a master server at 127.22.13.126:36381
I20260928 14:38:48.140807 816 heartbeater.cc:464] Registering TS with master...
I20260928 14:38:48.140988 816 heartbeater.cc:511] Master 127.22.13.126:36381 requested a full tablet report, sending...
I20260928 14:38:48.141410 352 ts_manager.cc:194] Registered new tserver with Master: 1af134aceb0241f0990869c6a801980d (127.22.13.67:33075)
I20260928 14:38:48.141788 352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:48103
I20260928 14:38:48.144124 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:38:48.151434 410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:48.145560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60370 (local address 127.22.13.126:36381)
0928 14:38:48.145633 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:48.145635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:48.145723 (+ 88us) server_negotiation.cc:407] Connection header received
0928 14:38:48.145798 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:48.145799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:48.145808 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:48.145871 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:38:48.146477 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.146963 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.147688 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.147906 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.147982 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:48.149102 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:38:48.149104 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:38:48.149104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:38:48.149112 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:38:48.150284 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:48.150652 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:48.150653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:48.150653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:48.150710 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:38:48.150901 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:38:48.150902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:38:48.150902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:38:48.151053 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:38:48.151160 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:48.151254 (+ 94us) server_negotiation.cc:299] Negotiation successful
0928 14:38:48.151334 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 14:38:48.153205 352 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:38:48.154893 352 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:38:48.161562 826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:48.157020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:41039 (local address 127.0.0.1:38256)
0928 14:38:48.157617 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:48.157658 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:48.157733 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:48.158106 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:48.158109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:48.158111 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:48.158187 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:48.158189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.159167 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.159168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:48.160277 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:48.160279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.161222 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.161223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:48.161308 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:48.161323 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 14:38:48.161405 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 14:38:48.161461 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":66,"threads_started":1}
I20260928 14:38:48.161662 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:48.157020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:42603 (local address 127.0.0.1:53110)
0928 14:38:48.157875 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:48.157883 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:48.157931 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:48.158382 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:48.158383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:48.158386 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:48.158428 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:48.158430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.159669 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.159670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:48.160655 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:48.160657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.161382 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.161383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:48.161446 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:48.161451 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:38:48.161505 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 14:38:48.161547 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"mutex_wait_us":102,"thread_start_us":65,"threads_started":1}
I20260928 14:38:48.161845 825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:48.157040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:33075 (local address 127.0.0.1:47318)
0928 14:38:48.157982 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:48.157992 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:48.158041 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:48.158277 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:48.158278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:48.158279 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:48.158310 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:48.158317 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.159672 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.159673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:48.160699 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:48.160702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.161633 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.161634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:48.161696 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:48.161700 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:38:48.161745 (+ 45us) client_negotiation.cc:240] Negotiation successful
0928 14:38:48.161787 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"mutex_wait_us":2,"thread_start_us":69,"threads_started":1}
I20260928 14:38:48.161899 829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:48.157425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38256 (local address 127.22.13.66:41039)
0928 14:38:48.157594 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:48.157597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:48.157737 (+ 140us) server_negotiation.cc:407] Connection header received
0928 14:38:48.157853 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:48.157856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:48.157882 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:48.158016 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:48.158254 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.159066 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.160366 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.161073 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.161137 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:48.161190 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:48.161781 (+ 591us) server_negotiation.cc:299] Negotiation successful
0928 14:38:48.161822 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":95,"threads_started":1}
I20260928 14:38:48.162055 827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:48.157232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53110 (local address 127.22.13.65:42603)
0928 14:38:48.157426 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:48.157432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:48.157932 (+ 500us) server_negotiation.cc:407] Connection header received
0928 14:38:48.158201 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:48.158204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:48.158220 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:48.158293 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:48.158508 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.159572 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.160735 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.161283 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.161379 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:48.161433 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:48.161522 (+ 89us) server_negotiation.cc:299] Negotiation successful
0928 14:38:48.161556 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":78,"threads_started":1}
I20260928 14:38:48.162120 828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:48.157281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47318 (local address 127.22.13.67:33075)
0928 14:38:48.157592 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:48.157597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:48.158039 (+ 442us) server_negotiation.cc:407] Connection header received
0928 14:38:48.158112 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:48.158115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:48.158131 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:48.158207 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:48.158401 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.159548 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.160793 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.161492 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.161551 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:48.161598 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:48.162000 (+ 402us) server_negotiation.cc:299] Negotiation successful
0928 14:38:48.162048 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":119,"threads_started":1}
I20260928 14:38:48.162793 478 tablet_service.cc:1467] Processing CreateTablet for tablet f17ccb9ebe1549b1b2689672665a500b (DEFAULT_TABLE table=test-table [id=bcbf987a5c224d44b6062cc63591ee00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:48.162853 613 tablet_service.cc:1467] Processing CreateTablet for tablet f17ccb9ebe1549b1b2689672665a500b (DEFAULT_TABLE table=test-table [id=bcbf987a5c224d44b6062cc63591ee00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:48.163115 613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f17ccb9ebe1549b1b2689672665a500b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:48.163121 478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f17ccb9ebe1549b1b2689672665a500b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:48.165794 830 tablet_bootstrap.cc:492] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5: Bootstrap starting.
I20260928 14:38:48.166507 830 tablet_bootstrap.cc:654] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:48.166401 749 tablet_service.cc:1467] Processing CreateTablet for tablet f17ccb9ebe1549b1b2689672665a500b (DEFAULT_TABLE table=test-table [id=bcbf987a5c224d44b6062cc63591ee00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:38:48.166731 749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f17ccb9ebe1549b1b2689672665a500b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:38:48.167084 830 log.cc:824] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:48.167865 830 tablet_bootstrap.cc:492] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5: No bootstrap required, opened a new log
I20260928 14:38:48.167927 830 ts_tablet_manager.cc:1397] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:38:48.168190 831 tablet_bootstrap.cc:492] T f17ccb9ebe1549b1b2689672665a500b P a35e17b8fb104927be8236cf7f5b4cde: Bootstrap starting.
I20260928 14:38:48.168877 831 tablet_bootstrap.cc:654] T f17ccb9ebe1549b1b2689672665a500b P a35e17b8fb104927be8236cf7f5b4cde: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:48.169413 831 log.cc:824] T f17ccb9ebe1549b1b2689672665a500b P a35e17b8fb104927be8236cf7f5b4cde: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:48.170032 834 tablet_bootstrap.cc:492] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d: Bootstrap starting.
I20260928 14:38:48.170022 830 raft_consensus.cc:348] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0c5b657e7143aa8ade5cc148f30ad5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41039 } } peers { permanent_uuid: "a35e17b8fb104927be8236cf7f5b4cde" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42603 } } peers { permanent_uuid: "1af134aceb0241f0990869c6a801980d" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33075 } }
I20260928 14:38:48.170181 830 raft_consensus.cc:374] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:48.170194 831 tablet_bootstrap.cc:492] T f17ccb9ebe1549b1b2689672665a500b P a35e17b8fb104927be8236cf7f5b4cde: No bootstrap required, opened a new log
I20260928 14:38:48.170215 830 raft_consensus.cc:729] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0c5b657e7143aa8ade5cc148f30ad5, State: Initialized, Role: FOLLOWER
I20260928 14:38:48.170253 831 ts_tablet_manager.cc:1397] T f17ccb9ebe1549b1b2689672665a500b P a35e17b8fb104927be8236cf7f5b4cde: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:38:48.170346 830 consensus_queue.cc:260] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0c5b657e7143aa8ade5cc148f30ad5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41039 } } peers { permanent_uuid: "a35e17b8fb104927be8236cf7f5b4cde" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42603 } } peers { permanent_uuid: "1af134aceb0241f0990869c6a801980d" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33075 } }
I20260928 14:38:48.170603 681 heartbeater.cc:503] Master 127.22.13.126:36381 was elected leader, sending a full tablet report...
I20260928 14:38:48.170667 834 tablet_bootstrap.cc:654] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d: Neither blocks nor log segments found. Creating new log.
I20260928 14:38:48.170846 830 ts_tablet_manager.cc:1428] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:38:48.171190 834 log.cc:824] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:38:48.172175 834 tablet_bootstrap.cc:492] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d: No bootstrap required, opened a new log
I20260928 14:38:48.172250 834 ts_tablet_manager.cc:1397] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:38:48.172446 831 raft_consensus.cc:348] T f17ccb9ebe1549b1b2689672665a500b P a35e17b8fb104927be8236cf7f5b4cde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0c5b657e7143aa8ade5cc148f30ad5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41039 } } peers { permanent_uuid: "a35e17b8fb104927be8236cf7f5b4cde" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42603 } } peers { permanent_uuid: "1af134aceb0241f0990869c6a801980d" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33075 } }
I20260928 14:38:48.172667 831 raft_consensus.cc:374] T f17ccb9ebe1549b1b2689672665a500b P a35e17b8fb104927be8236cf7f5b4cde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:48.172727 831 raft_consensus.cc:729] T f17ccb9ebe1549b1b2689672665a500b P a35e17b8fb104927be8236cf7f5b4cde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a35e17b8fb104927be8236cf7f5b4cde, State: Initialized, Role: FOLLOWER
I20260928 14:38:48.172843 831 consensus_queue.cc:260] T f17ccb9ebe1549b1b2689672665a500b P a35e17b8fb104927be8236cf7f5b4cde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0c5b657e7143aa8ade5cc148f30ad5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41039 } } peers { permanent_uuid: "a35e17b8fb104927be8236cf7f5b4cde" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42603 } } peers { permanent_uuid: "1af134aceb0241f0990869c6a801980d" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33075 } }
I20260928 14:38:48.173106 831 ts_tablet_manager.cc:1428] T f17ccb9ebe1549b1b2689672665a500b P a35e17b8fb104927be8236cf7f5b4cde: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:38:48.173130 545 heartbeater.cc:503] Master 127.22.13.126:36381 was elected leader, sending a full tablet report...
I20260928 14:38:48.173918 834 raft_consensus.cc:348] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0c5b657e7143aa8ade5cc148f30ad5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41039 } } peers { permanent_uuid: "a35e17b8fb104927be8236cf7f5b4cde" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42603 } } peers { permanent_uuid: "1af134aceb0241f0990869c6a801980d" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33075 } }
I20260928 14:38:48.174109 834 raft_consensus.cc:374] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:38:48.174134 834 raft_consensus.cc:729] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1af134aceb0241f0990869c6a801980d, State: Initialized, Role: FOLLOWER
I20260928 14:38:48.174263 834 consensus_queue.cc:260] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0c5b657e7143aa8ade5cc148f30ad5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41039 } } peers { permanent_uuid: "a35e17b8fb104927be8236cf7f5b4cde" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42603 } } peers { permanent_uuid: "1af134aceb0241f0990869c6a801980d" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33075 } }
I20260928 14:38:48.174520 834 ts_tablet_manager.cc:1428] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260928 14:38:48.174546 816 heartbeater.cc:503] Master 127.22.13.126:36381 was elected leader, sending a full tablet report...
W20260928 14:38:48.240003 682 tablet.cc:2369] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:38:48.252831 838 raft_consensus.cc:482] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:38:48.253023 838 raft_consensus.cc:504] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0c5b657e7143aa8ade5cc148f30ad5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41039 } } peers { permanent_uuid: "a35e17b8fb104927be8236cf7f5b4cde" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42603 } } peers { permanent_uuid: "1af134aceb0241f0990869c6a801980d" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33075 } }
I20260928 14:38:48.253459 838 leader_election.cc:288] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b0c5b657e7143aa8ade5cc148f30ad5 (127.22.13.66:41039), a35e17b8fb104927be8236cf7f5b4cde (127.22.13.65:42603)
I20260928 14:38:48.256273 829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:48.253746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:39221 (local address 127.22.13.66:41039)
0928 14:38:48.253798 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:48.253800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:48.253808 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:38:48.253836 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:48.253839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:48.253855 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:48.253925 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:48.254100 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.254574 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.255535 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.255921 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.256007 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:48.256016 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:48.256161 (+ 145us) server_negotiation.cc:299] Negotiation successful
0928 14:38:48.256201 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260928 14:38:48.256273 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:48.253650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:41039 (local address 127.22.13.67:39221)
0928 14:38:48.253706 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:48.253718 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:48.253762 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:48.253952 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:48.253953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:48.253954 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:48.254027 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:48.254029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.254670 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.254671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:48.255432 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:48.255434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.256006 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.256007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:48.256087 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:48.256091 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:38:48.256155 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 14:38:48.256201 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260928 14:38:48.256464 827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:48.253742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:60537 (local address 127.22.13.65:42603)
0928 14:38:48.253788 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:48.253791 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:48.253959 (+ 168us) server_negotiation.cc:407] Connection header received
0928 14:38:48.254046 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:48.254047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:48.254060 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:48.254127 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:38:48.254292 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.254750 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.255681 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.256068 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.256155 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:48.256165 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:48.256321 (+ 156us) server_negotiation.cc:299] Negotiation successful
0928 14:38:48.256366 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:38:48.256652 839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:48.253680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:42603 (local address 127.22.13.67:60537)
0928 14:38:48.253868 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:48.253877 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:38:48.253973 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:38:48.254145 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:38:48.254146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:38:48.254147 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:38:48.254220 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:48.254221 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.254838 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.254839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:48.255580 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:38:48.255582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.256164 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.256166 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:38:48.256240 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:48.256243 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:38:48.256302 (+ 59us) client_negotiation.cc:240] Negotiation successful
0928 14:38:48.256489 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":1,"thread_start_us":110,"threads_started":1}
I20260928 14:38:48.256609 633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f17ccb9ebe1549b1b2689672665a500b" candidate_uuid: "1af134aceb0241f0990869c6a801980d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0c5b657e7143aa8ade5cc148f30ad5" is_pre_election: true
I20260928 14:38:48.256778 633 raft_consensus.cc:2454] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1af134aceb0241f0990869c6a801980d in term 0.
I20260928 14:38:48.256974 499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f17ccb9ebe1549b1b2689672665a500b" candidate_uuid: "1af134aceb0241f0990869c6a801980d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a35e17b8fb104927be8236cf7f5b4cde" is_pre_election: true
I20260928 14:38:48.257004 703 leader_election.cc:302] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b0c5b657e7143aa8ade5cc148f30ad5, 1af134aceb0241f0990869c6a801980d; no voters:
I20260928 14:38:48.257134 499 raft_consensus.cc:2454] T f17ccb9ebe1549b1b2689672665a500b P a35e17b8fb104927be8236cf7f5b4cde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1af134aceb0241f0990869c6a801980d in term 0.
I20260928 14:38:48.257138 838 raft_consensus.cc:2781] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:38:48.257195 838 raft_consensus.cc:482] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:38:48.257221 838 raft_consensus.cc:3037] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:48.257979 838 raft_consensus.cc:504] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0c5b657e7143aa8ade5cc148f30ad5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41039 } } peers { permanent_uuid: "a35e17b8fb104927be8236cf7f5b4cde" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42603 } } peers { permanent_uuid: "1af134aceb0241f0990869c6a801980d" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33075 } }
I20260928 14:38:48.258124 838 leader_election.cc:288] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [CANDIDATE]: Term 1 election: Requested vote from peers 0b0c5b657e7143aa8ade5cc148f30ad5 (127.22.13.66:41039), a35e17b8fb104927be8236cf7f5b4cde (127.22.13.65:42603)
I20260928 14:38:48.258296 633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f17ccb9ebe1549b1b2689672665a500b" candidate_uuid: "1af134aceb0241f0990869c6a801980d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0c5b657e7143aa8ade5cc148f30ad5"
I20260928 14:38:48.258317 499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f17ccb9ebe1549b1b2689672665a500b" candidate_uuid: "1af134aceb0241f0990869c6a801980d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a35e17b8fb104927be8236cf7f5b4cde"
I20260928 14:38:48.258396 633 raft_consensus.cc:3037] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:48.258399 499 raft_consensus.cc:3037] T f17ccb9ebe1549b1b2689672665a500b P a35e17b8fb104927be8236cf7f5b4cde [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:38:48.259138 499 raft_consensus.cc:2454] T f17ccb9ebe1549b1b2689672665a500b P a35e17b8fb104927be8236cf7f5b4cde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1af134aceb0241f0990869c6a801980d in term 1.
I20260928 14:38:48.259138 633 raft_consensus.cc:2454] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1af134aceb0241f0990869c6a801980d in term 1.
I20260928 14:38:48.259321 705 leader_election.cc:302] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1af134aceb0241f0990869c6a801980d, a35e17b8fb104927be8236cf7f5b4cde; no voters:
I20260928 14:38:48.259438 838 raft_consensus.cc:2781] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:38:48.259609 838 raft_consensus.cc:686] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [term 1 LEADER]: Becoming Leader. State: Replica: 1af134aceb0241f0990869c6a801980d, State: Running, Role: LEADER
I20260928 14:38:48.259718 838 consensus_queue.cc:237] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0c5b657e7143aa8ade5cc148f30ad5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41039 } } peers { permanent_uuid: "a35e17b8fb104927be8236cf7f5b4cde" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42603 } } peers { permanent_uuid: "1af134aceb0241f0990869c6a801980d" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33075 } }
I20260928 14:38:48.260627 352 catalog_manager.cc:5495] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1af134aceb0241f0990869c6a801980d (127.22.13.67). New cstate: current_term: 1 leader_uuid: "1af134aceb0241f0990869c6a801980d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b0c5b657e7143aa8ade5cc148f30ad5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a35e17b8fb104927be8236cf7f5b4cde" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1af134aceb0241f0990869c6a801980d" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33075 } health_report { overall_health: HEALTHY } } }
I20260928 14:38:48.277240 828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:48.274720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47328 (local address 127.22.13.67:33075)
0928 14:38:48.274784 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 14:38:48.274786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:38:48.274795 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:38:48.274837 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:38:48.274838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:38:48.274862 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:38:48.274942 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:38:48.275099 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.275587 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.276372 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:38:48.276582 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:38:48.276660 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:38:48.276842 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:38:48.276962 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:38:48.277053 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 14:38:48.277103 (+ 50us) server_negotiation.cc:299] Negotiation successful
0928 14:38:48.277124 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 14:38:48.279320 499 raft_consensus.cc:1261] T f17ccb9ebe1549b1b2689672665a500b P a35e17b8fb104927be8236cf7f5b4cde [term 1 FOLLOWER]: Refusing update from remote peer 1af134aceb0241f0990869c6a801980d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:48.279320 633 raft_consensus.cc:1261] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5 [term 1 FOLLOWER]: Refusing update from remote peer 1af134aceb0241f0990869c6a801980d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:38:48.279731 840 consensus_queue.cc:1035] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b0c5b657e7143aa8ade5cc148f30ad5" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:48.279933 838 consensus_queue.cc:1035] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a35e17b8fb104927be8236cf7f5b4cde" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:38:48.283414 845 mvcc.cc:205] Tried to move back new op lower bound from 7334323520629018624 to 7334323520552800256. Current Snapshot: MvccSnapshot[applied={T|T < 7334323520629018624 or (T in {7334323520629018624})}]
I20260928 14:38:48.284545 352 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:48.284652 352 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:38:48.285411 352 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 843650b315ac4bb0a708b8e4eb16f0c6: Sending DeleteTablet for 3 replicas of tablet f17ccb9ebe1549b1b2689672665a500b
I20260928 14:38:48.285771 749 tablet_service.cc:1514] Processing DeleteTablet for tablet f17ccb9ebe1549b1b2689672665a500b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:48 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:47318
I20260928 14:38:48.286028 478 tablet_service.cc:1514] Processing DeleteTablet for tablet f17ccb9ebe1549b1b2689672665a500b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:48 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:53110
I20260928 14:38:48.286072 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 415
I20260928 14:38:48.286764 613 tablet_service.cc:1514] Processing DeleteTablet for tablet f17ccb9ebe1549b1b2689672665a500b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:38:48 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:38256
I20260928 14:38:48.286839 852 tablet_replica.cc:330] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d: stopping tablet replica
I20260928 14:38:48.286949 852 raft_consensus.cc:2229] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:38:48.287077 852 raft_consensus.cc:2258] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:48.287457 852 ts_tablet_manager.cc:1905] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:48.289309 852 ts_tablet_manager.cc:1918] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:38:48.289400 852 log.cc:1196] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/f17ccb9ebe1549b1b2689672665a500b
I20260928 14:38:48.289659 852 ts_tablet_manager.cc:1939] T f17ccb9ebe1549b1b2689672665a500b P 1af134aceb0241f0990869c6a801980d: Deleting consensus metadata
I20260928 14:38:48.290081 854 tablet_replica.cc:330] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5: stopping tablet replica
I20260928 14:38:48.290109 338 catalog_manager.cc:4841] TS 1af134aceb0241f0990869c6a801980d (127.22.13.67:33075): tablet f17ccb9ebe1549b1b2689672665a500b (table test-table [id=bcbf987a5c224d44b6062cc63591ee00]) successfully deleted
I20260928 14:38:48.290295 854 raft_consensus.cc:2229] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:38:48.290455 854 raft_consensus.cc:2258] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:38:48.290886 854 ts_tablet_manager.cc:1905] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:38:48.292135 854 ts_tablet_manager.cc:1918] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:38:48.292230 854 log.cc:1196] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/f17ccb9ebe1549b1b2689672665a500b
I20260928 14:38:48.292501 854 ts_tablet_manager.cc:1939] T f17ccb9ebe1549b1b2689672665a500b P 0b0c5b657e7143aa8ade5cc148f30ad5: Deleting consensus metadata
I20260928 14:38:48.292925 337 catalog_manager.cc:4841] TS 0b0c5b657e7143aa8ade5cc148f30ad5 (127.22.13.66:41039): tablet f17ccb9ebe1549b1b2689672665a500b (table test-table [id=bcbf987a5c224d44b6062cc63591ee00]) successfully deleted
W20260928 14:38:48.293426 339 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:42603 (error 108)
I20260928 14:38:48.293696 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 550
I20260928 14:38:48.293893 825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:38:48.293764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:42603 (local address 127.0.0.1:53114)
0928 14:38:48.293814 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 14:38:48.293845 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:42603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260928 14:38:48.294404 339 catalog_manager.cc:4584] TS a35e17b8fb104927be8236cf7f5b4cde (127.22.13.65:42603): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f17ccb9ebe1549b1b2689672665a500b: Network error: Client connection negotiation failed: client connection to 127.22.13.65:42603: connect: Connection refused (error 111)
I20260928 14:38:48.298589 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 686
I20260928 14:38:48.303257 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 320
2026-09-28T14:38:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1942 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:38:48 dist-test-slave-wwwb krb5kdc[858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:38:48 dist-test-slave-wwwb krb5kdc[858](info): set up 2 sockets
Sep 28 14:38:48 dist-test-slave-wwwb krb5kdc[858](info): commencing operation
krb5kdc: starting...
W20260928 14:38:49.327461 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:38:49 dist-test-slave-wwwb krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:38:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:38:49Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.13.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.13.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.13.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/rangeradmin_127.22.13.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.13.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/rangeradmin_127.22.13.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.13.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.13.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.13.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/rangerlookup_127.22.13.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.13.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/rangerlookup_127.22.13.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.13.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.13.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.13.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/HTTP_127.22.13.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.13.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/HTTP_127.22.13.84@KRBTEST.COM.keytab.
I20260928 14:38:49.488866 22581 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskT1ZEJk/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/postgres -l logfile start
2026-09-28 14:38:51.927 UTC [888] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-28 14:38:51.927 UTC [888] LOG: listening on IPv4 address "127.22.13.84", port 41533
2026-09-28 14:38:51.930 UTC [888] LOG: listening on Unix socket "/tmp/.s.PGSQL.41533"
2026-09-28 14:38:51.964 UTC [891] LOG: database system was shut down at 2026-09-28 14:38:50 UTC
2026-09-28 14:38:51.968 UTC [888] LOG: database system is ready to accept connections
I20260928 14:38:52.915716 22581 mini_postgres.cc:96] Postgres bound to 41533
2026-09-28 14:38:52.922 UTC [899] FATAL: database "slave" does not exist
127.22.13.84:41533 - accepting connections
I20260928 14:38:52.922850 22581 mini_ranger.cc:161] Starting Ranger...
I20260928 14:38:52.938542 22581 mini_ranger.cc:84] Created miniranger Postgres user
I20260928 14:38:53.365643 22581 mini_ranger.cc:87] Created ranger Postgres database
I20260928 14:38:53.365732 22581 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-admin
2026-09-28 14:38:53,661 [I] DB FLAVOR :POSTGRES
2026-09-28 14:38:53,662 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 14:38:53,662 [I] Checking connection..
2026-09-28 14:38:53,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-28 14:38:53,925 [I] Checking connection passed.
2026-09-28 14:38:53,925 [I] --------- Verifying version history table ---------
2026-09-28 14:38:53,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:38:54,194 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 14:38:54,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:38:54,463 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 14:38:54,463 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-28 14:38:54,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-28 14:38:54.728 UTC [963] WARNING: there is no transaction in progress
2026-09-28 14:38:54,735 [I] create_dbversion_catalog.sql file imported successfully
2026-09-28 14:38:54,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:38:55,011 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-28 14:38:55,011 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 14:38:55,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-28 14:38:55,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-28 14:38:55,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-wwwb.c.gcp-upstream.internal','N') ;"
2026-09-28 14:38:55,766 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 14:38:56,036 [I] Table x_portal_user does not exist in database ranger
2026-09-28 14:38:56,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 14:38:56,304 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-28 14:38:56,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 14:38:56,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 14:38:56,804 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-28 14:38:56,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-28 14:38:57.760 UTC [1073] WARNING: there is no transaction in progress
2026-09-28 14:38:57.774 UTC [1073] WARNING: there is no transaction in progress
2026-09-28 14:38:57.790 UTC [1073] WARNING: there is no transaction in progress
2026-09-28 14:38:57.805 UTC [1073] WARNING: there is no transaction in progress
2026-09-28 14:38:57.819 UTC [1073] WARNING: there is no transaction in progress
2026-09-28 14:38:57.836 UTC [1073] WARNING: there is no transaction in progress
2026-09-28 14:38:57.893 UTC [1073] WARNING: there is no transaction in progress
2026-09-28 14:38:57.899 UTC [1073] WARNING: there is no transaction in progress
2026-09-28 14:38:57.913 UTC [1073] WARNING: there is no transaction in progress
2026-09-28 14:38:57.922 UTC [1073] WARNING: there is no transaction in progress
2026-09-28 14:38:57.932 UTC [1073] WARNING: there is no transaction in progress
2026-09-28 14:38:57.946 UTC [1073] WARNING: there is no transaction in progress
2026-09-28 14:38:57.955 UTC [1073] WARNING: there is no transaction in progress
2026-09-28 14:38:57.962 UTC [1073] WARNING: there is no transaction in progress
2026-09-28 14:38:57.968 UTC [1073] WARNING: there is no transaction in progress
2026-09-28 14:38:57.974 UTC [1073] WARNING: there is no transaction in progress
2026-09-28 14:38:58,563 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-28 14:38:58,564 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-28 14:38:58,831 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-28 14:38:58,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 14:38:59,074 [I] Table x_portal_user already exists in database 'ranger'
2026-09-28 14:38:59,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 14:38:59,326 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-28 14:38:59,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 14:38:59,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 14:38:59,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-wwwb.c.gcp-upstream.internal';"
2026-09-28 14:39:00,065 [I] CORE_DB_SCHEMA import status has been updated
2026-09-28 14:39:00,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-28 14:39:00,331 [I] DB_PATCHES have already been applied
I20260928 14:39:00.336927 22581 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-admin:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskT1ZEJk/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260928 14:39:00.337030 22581 mini_ranger.cc:193] Using host: 127.22.13.84
I20260928 14:39:00.340186 22581 mini_ranger.cc:227] Ranger admin URL: http://127.22.13.84:40865
Sep 28, 2026 2:39:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 2:39:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 2:39:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-admin/ews/webapp
Sep 28, 2026 2:39:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 28, 2026 2:39:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 28, 2026 2:39:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-admin/ews/webapp].
Sep 28, 2026 2:39:01 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260928 14:39:01.437879 22581 mini_ranger.cc:160] Time spent starting Ranger: real 8.515s user 0.004s sys 0.006s
Sep 28, 2026 2:39:23 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 28, 2026 2:39:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 28, 2026 2:39:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 28, 2026 2:39:23 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 28, 2026 2:39:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 28, 2026 2:39:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 28, 2026 2:39:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 28, 2026 2:39:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 28, 2026 2:39:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 28, 2026 2:39:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 28, 2026 2:39:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 28, 2026 2:39:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 28, 2026 2:39:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 28, 2026 2:39:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 28, 2026 2:39:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 28, 2026 2:39:23 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 28, 2026 2:39:23 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 28, 2026 2:39:24 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260928 14:39:24.659749 22581 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.13.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.13.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.13.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/rangerkms_127.22.13.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.13.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/rangerkms_127.22.13.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.13.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.13.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.13.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/HTTP_127.22.13.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.13.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/HTTP_127.22.13.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 28 14:39:24 dist-test-slave-wwwb krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606364, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260928 14:39:24.759917 22581 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260928 14:39:24.770620 22581 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260928 14:39:25.556905 22581 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260928 14:39:25.556998 22581 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-kms
I20260928 14:39:25.557016 22581 mini_ranger_kms.cc:227] Using postgres at 127.22.13.84:41533
2026-09-28 14:39:25,967 [I] DB FLAVOR :POSTGRES
2026-09-28 14:39:25,968 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 14:39:25,968 [I] Checking connection
2026-09-28 14:39:25,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-28 14:39:26,265 [I] connection success
2026-09-28 14:39:26,265 [I] --------- Verifying Ranger DB tables ---------
2026-09-28 14:39:26,266 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-28 14:39:26,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-28 14:39:26,585 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-28 14:39:26,585 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 14:39:26,586 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-28 14:39:26,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:41533/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-28 14:39:26,899 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260928 14:39:27.442026 22581 mini_ranger_kms.cc:310] Created kms service
I20260928 14:39:27.667691 22581 mini_ranger_kms.cc:326] Created kudu user
I20260928 14:39:27.722743 22581 mini_ranger_kms.cc:343] Created rangerkms user
Sep 28 14:39:27 dist-test-slave-wwwb krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606364, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:39:28.209389 22581 mini_ranger_kms.cc:384] Added ranger policy
I20260928 14:39:28.209506 22581 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-kms:/tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260928 14:39:28.209529 22581 mini_ranger_kms.cc:240] Using host: 127.22.13.84
I20260928 14:39:28.213456 22581 mini_ranger_kms.cc:276] Ranger KMS PID: 1255
I20260928 14:39:28.213621 22581 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.13.84:51315
Sep 28, 2026 2:39:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 2:39:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 2:39:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 28, 2026 2:39:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 28, 2026 2:39:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-kms/ews/webapp].
Sep 28, 2026 2:39:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260928 14:39:30.458250 22581 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.698s user 0.000s sys 0.246s
I20260928 14:39:30.458374 22581 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260928 14:39:30.458451 22581 mini_ranger_kms.cc:401] 127.22.13.84:51315/kms/v1/keys
Sep 28 14:39:30 dist-test-slave-wwwb krb5kdc[858](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790606370, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.13.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 28 14:39:31 dist-test-slave-wwwb krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606370, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.13.84@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
Sep 28 14:39:31 dist-test-slave-wwwb krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606370, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.13.84@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
Sep 28 14:39:33 dist-test-slave-wwwb krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:39:34.025667 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:40829
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:42567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:40829
--ranger_config_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.13.84:51315/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:39:34.111207 1305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:39:34.111438 1305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:39:34.111462 1305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:39:34.112708 1305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:39:34.112757 1305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:39:34.112772 1305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:39:34.112785 1305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:39:34.112797 1305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:39:34.114568 1305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:42567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.13.84:51315/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:40829
--ranger_config_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:40829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:39:34.114845 1305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:39:34.115095 1305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:39:34.120071 1311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:39:34.120078 1313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:39:34.120085 1310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:39:34.120388 1305 server_base.cc:1034] running on GCE node
I20260928 14:39:34.120654 1305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:39:34.120987 1305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:39:34.122150 1305 hybrid_clock.cc:648] HybridClock initialized: now 1790606374122127 us; error 42 us; skew 500 ppm
Sep 28 14:39:34 dist-test-slave-wwwb krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606374, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:39:34.128046 1305 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:39:34.128629 1305 webserver.cc:460] Webserver started at http://127.22.13.126:36251/ using document root <none> and password file <none>
I20260928 14:39:34.128899 1305 fs_manager.cc:359] Metadata directory not provided
I20260928 14:39:34.128963 1305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:39:34.129124 1305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:39:34 dist-test-slave-wwwb krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606374, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:39:34.191949 1305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "8523c484769a4930bbfa89054c5a626b"
format_stamp: "Formatted at 2026-09-28 14:39:34 on dist-test-slave-wwwb"
server_key: "0c94d398e7cdd040bbcac7c900ce2460"
server_key_iv: "0457eb5da5a30448a1141ffe538fe9fc"
server_key_version: "kuduclusterkey@0"
I20260928 14:39:34.192353 1305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "8523c484769a4930bbfa89054c5a626b"
format_stamp: "Formatted at 2026-09-28 14:39:34 on dist-test-slave-wwwb"
server_key: "0c94d398e7cdd040bbcac7c900ce2460"
server_key_iv: "0457eb5da5a30448a1141ffe538fe9fc"
server_key_version: "kuduclusterkey@0"
I20260928 14:39:34.193859 1305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:39:34.210495 1320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:39:34.210712 1305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:39:34.210809 1305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "8523c484769a4930bbfa89054c5a626b"
format_stamp: "Formatted at 2026-09-28 14:39:34 on dist-test-slave-wwwb"
server_key: "0c94d398e7cdd040bbcac7c900ce2460"
server_key_iv: "0457eb5da5a30448a1141ffe538fe9fc"
server_key_version: "kuduclusterkey@0"
I20260928 14:39:34.210916 1305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:39:34.238920 1305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:39:34.241338 1305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:39:34.241515 1305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:39:34.247051 1305 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:40829
I20260928 14:39:34.247094 1373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:40829 every 8 connection(s)
I20260928 14:39:34.247606 1305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:39:34.248411 1374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:39:34.250895 1374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b: Bootstrap starting.
I20260928 14:39:34.251523 1374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b: Neither blocks nor log segments found. Creating new log.
I20260928 14:39:34.251849 1374 log.cc:824] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:39:34.252450 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 1305
I20260928 14:39:34.252581 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:39:34.252732 1374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b: No bootstrap required, opened a new log
I20260928 14:39:34.252857 22581 external_mini_cluster.cc:1428] Setting key 26bef9b2cde7fa6a91e0ede32ae40e4a
I20260928 14:39:34.255990 1374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8523c484769a4930bbfa89054c5a626b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40829 } }
I20260928 14:39:34.256244 1374 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:39:34.256315 1374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8523c484769a4930bbfa89054c5a626b, State: Initialized, Role: FOLLOWER
I20260928 14:39:34.256471 1374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8523c484769a4930bbfa89054c5a626b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40829 } }
I20260928 14:39:34.256601 1374 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:39:34.256652 1374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:39:34.256735 1374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:39:34.257587 1374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8523c484769a4930bbfa89054c5a626b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40829 } }
I20260928 14:39:34.257741 1374 leader_election.cc:302] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8523c484769a4930bbfa89054c5a626b; no voters:
I20260928 14:39:34.258004 1374 leader_election.cc:288] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:39:34.258245 1380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:39:34.258311 1374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:39:34.259550 1380 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [term 1 LEADER]: Becoming Leader. State: Replica: 8523c484769a4930bbfa89054c5a626b, State: Running, Role: LEADER
I20260928 14:39:34.259727 1380 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8523c484769a4930bbfa89054c5a626b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40829 } }
Sep 28 14:39:34 dist-test-slave-wwwb krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:39:34.260686 1380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8523c484769a4930bbfa89054c5a626b. Latest consensus state: current_term: 1 leader_uuid: "8523c484769a4930bbfa89054c5a626b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8523c484769a4930bbfa89054c5a626b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40829 } } }
I20260928 14:39:34.260864 1380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [sys.catalog]: This master's current role is: LEADER
I20260928 14:39:34.261078 1374 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 14:39:34.261333 1379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8523c484769a4930bbfa89054c5a626b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8523c484769a4930bbfa89054c5a626b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40829 } } }
I20260928 14:39:34.261400 1379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b [sys.catalog]: This master's current role is: LEADER
I20260928 14:39:34.261140 1393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:39:34.261874 1393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:39:34.269104 1378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:34.253412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42226 (local address 127.22.13.126:40829)
0928 14:39:34.253624 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 14:39:34.253635 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:39:34.253666 (+ 31us) server_negotiation.cc:407] Connection header received
0928 14:39:34.254270 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:39:34.254278 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:39:34.254338 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:39:34.254495 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:39:34.255149 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:34.256060 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:34.257131 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:34.257571 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:34.257776 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:34.260551 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:39:34.260556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:39:34.260557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:39:34.260572 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:39:34.266287 (+ 5715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:39:34.266830 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:39:34.266834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:39:34.266834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:39:34.266957 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:39:34.267227 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:39:34.267228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:39:34.267228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:39:34.267777 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:39:34.267965 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0928 14:39:34.268609 (+ 644us) server_negotiation.cc:299] Negotiation successful
0928 14:39:34.268812 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":87,"threads_started":1}
I20260928 14:39:34.273520 1393 catalog_manager.cc:1347] Generated new cluster ID: f587ce2efdd1421fa824271449091f12
I20260928 14:39:34.273605 1393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:39:34.284986 1393 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:39:34.285835 1393 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:39:34.294122 1393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b: Generated new TSK 0
I20260928 14:39:34.294466 1393 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:39:35.032970 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.13.84:51315/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40829
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:42567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:39:35.158406 1419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:39:35.158643 1419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:39:35.158669 1419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:39:35.160121 1419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:39:35.160275 1419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:39:35.160332 1419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:39:35.162083 1419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:42567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.13.84:51315/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:39:35.162335 1419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:39:35.162602 1419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:39:35.167775 1425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:39:35.167912 1427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:39:35.167775 1424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:39:35.168569 1419 server_base.cc:1034] running on GCE node
I20260928 14:39:35.168792 1419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:39:35.169085 1419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:39:35.172384 1419 hybrid_clock.cc:648] HybridClock initialized: now 1790606375172340 us; error 51 us; skew 500 ppm
Sep 28 14:39:35 dist-test-slave-wwwb krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606375, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:39:35.175951 1419 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:39:35.176623 1419 webserver.cc:460] Webserver started at http://127.22.13.65:40245/ using document root <none> and password file <none>
I20260928 14:39:35.176913 1419 fs_manager.cc:359] Metadata directory not provided
I20260928 14:39:35.177008 1419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:39:35.177171 1419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:39:35 dist-test-slave-wwwb krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606375, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:39:35.196107 1419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "74b381a4f2664b7686636fcd6bac1046"
format_stamp: "Formatted at 2026-09-28 14:39:35 on dist-test-slave-wwwb"
server_key: "ff2ef98b5fe7ef2e7e09ea713c4a4374"
server_key_iv: "0fc4b8e8e327aa964a497e3b0aa01990"
server_key_version: "kuduclusterkey@0"
I20260928 14:39:35.196756 1419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "74b381a4f2664b7686636fcd6bac1046"
format_stamp: "Formatted at 2026-09-28 14:39:35 on dist-test-slave-wwwb"
server_key: "ff2ef98b5fe7ef2e7e09ea713c4a4374"
server_key_iv: "0fc4b8e8e327aa964a497e3b0aa01990"
server_key_version: "kuduclusterkey@0"
I20260928 14:39:35.199009 1419 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260928 14:39:35.214334 1434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:39:35.214790 1419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:39:35.214969 1419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "74b381a4f2664b7686636fcd6bac1046"
format_stamp: "Formatted at 2026-09-28 14:39:35 on dist-test-slave-wwwb"
server_key: "ff2ef98b5fe7ef2e7e09ea713c4a4374"
server_key_iv: "0fc4b8e8e327aa964a497e3b0aa01990"
server_key_version: "kuduclusterkey@0"
I20260928 14:39:35.215173 1419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:39:35.236127 1419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:39:35.238649 1419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:39:35.238863 1419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:39:35.239133 1419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:39:35.239593 1419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:39:35.239656 1419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:39:35.239691 1419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:39:35.239728 1419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:39:35.247886 1419 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:42875
I20260928 14:39:35.248498 1419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:39:35.249616 1549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:42875 every 8 connection(s)
I20260928 14:39:35.250694 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 1419
I20260928 14:39:35.250800 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:39:35.250995 22581 external_mini_cluster.cc:1428] Setting key d504d3a175cdc5045423c05b1660695e
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Sep 28 14:39:35 dist-test-slave-wwwb krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606375, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:39:35.270901 1553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.250396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:40829 (local address 127.22.13.65:42003)
0928 14:39:35.252753 (+ 2357us) negotiation.cc:107] Waiting for socket to connect
0928 14:39:35.252800 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:39:35.253437 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:39:35.253792 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:39:35.253797 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:39:35.254142 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:39:35.254450 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:39:35.254456 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.255538 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.255539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:39:35.257037 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:39:35.257041 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.257594 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.257595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:39:35.257765 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.261114 (+ 3349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:39:35.261129 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:39:35.266376 (+ 5247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:39:35.268754 (+ 2378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:39:35.268768 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:39:35.268771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:39:35.269171 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:39:35.269459 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:39:35.269460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:39:35.269461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:39:35.269654 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:39:35.270089 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:39:35.270098 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:39:35.270302 (+ 204us) client_negotiation.cc:769] Sending connection context
0928 14:39:35.270427 (+ 125us) client_negotiation.cc:240] Negotiation successful
0928 14:39:35.270614 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2306,"thread_start_us":90,"threads_started":1}
I20260928 14:39:35.271869 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.250394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:42003 (local address 127.22.13.126:40829)
0928 14:39:35.251294 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0928 14:39:35.251297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:39:35.252904 (+ 1607us) server_negotiation.cc:407] Connection header received
0928 14:39:35.253569 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:39:35.253571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:39:35.253585 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:39:35.254564 (+ 979us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:39:35.254613 (+ 49us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.255403 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.257159 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.257480 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.257565 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.266542 (+ 8977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:39:35.266545 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:39:35.266545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:39:35.266556 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:39:35.268621 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:39:35.269281 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:39:35.269283 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:39:35.269283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:39:35.269355 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:39:35.269760 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:39:35.269762 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:39:35.269762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:39:35.269929 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:39:35.270056 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0928 14:39:35.271653 (+ 1597us) server_negotiation.cc:299] Negotiation successful
0928 14:39:35.271759 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":866,"thread_start_us":103,"threads_started":1}
I20260928 14:39:35.272500 1550 heartbeater.cc:347] Connected to a master server at 127.22.13.126:40829
I20260928 14:39:35.272616 1550 heartbeater.cc:464] Registering TS with master...
I20260928 14:39:35.272835 1550 heartbeater.cc:511] Master 127.22.13.126:40829 requested a full tablet report, sending...
I20260928 14:39:35.273507 1338 ts_manager.cc:194] Registered new tserver with Master: 74b381a4f2664b7686636fcd6bac1046 (127.22.13.65:42875)
I20260928 14:39:35.274562 1338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:42003
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:39:35.284418 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.13.84:51315/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40829
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:42567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:39:35 dist-test-slave-wwwb krb5kdc[858](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790606375, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260928 14:39:35.406216 1556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:39:35.406584 1556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:39:35.406669 1556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:39:35.408593 1556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:39:35.408664 1556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:39:35.408710 1556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:39:35.411140 1556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:42567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.13.84:51315/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:39:35.411579 1556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:39:35.411897 1556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:39:35.418880 1566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:39:35.419346 1564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:39:35.420220 1563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:39:35.420703 1556 server_base.cc:1034] running on GCE node
I20260928 14:39:35.420956 1556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:39:35.421274 1556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:39:35.422447 1556 hybrid_clock.cc:648] HybridClock initialized: now 1790606375422428 us; error 30 us; skew 500 ppm
Sep 28 14:39:35 dist-test-slave-wwwb krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606375, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:39:35.426239 1556 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:39:35.426815 1556 webserver.cc:460] Webserver started at http://127.22.13.66:45283/ using document root <none> and password file <none>
I20260928 14:39:35.427098 1556 fs_manager.cc:359] Metadata directory not provided
I20260928 14:39:35.427151 1556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:39:35.427294 1556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:39:35 dist-test-slave-wwwb krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606375, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:39:35.444559 1556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "34d0603c30fb4539830e3424c89fde7f"
format_stamp: "Formatted at 2026-09-28 14:39:35 on dist-test-slave-wwwb"
server_key: "9e5ad64943ef564bbc79a7dea204061b"
server_key_iv: "a45422fda8f9ff26801faefd00dc568f"
server_key_version: "kuduclusterkey@0"
I20260928 14:39:35.445166 1556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "34d0603c30fb4539830e3424c89fde7f"
format_stamp: "Formatted at 2026-09-28 14:39:35 on dist-test-slave-wwwb"
server_key: "9e5ad64943ef564bbc79a7dea204061b"
server_key_iv: "a45422fda8f9ff26801faefd00dc568f"
server_key_version: "kuduclusterkey@0"
I20260928 14:39:35.449560 1556 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260928 14:39:35.462968 1574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:39:35.463243 1556 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:39:35.463332 1556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "34d0603c30fb4539830e3424c89fde7f"
format_stamp: "Formatted at 2026-09-28 14:39:35 on dist-test-slave-wwwb"
server_key: "9e5ad64943ef564bbc79a7dea204061b"
server_key_iv: "a45422fda8f9ff26801faefd00dc568f"
server_key_version: "kuduclusterkey@0"
I20260928 14:39:35.463428 1556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:39:35.474359 1556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:39:35.477209 1556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:39:35.477438 1556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:39:35.477756 1556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:39:35.478228 1556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:39:35.478325 1556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:39:35.478399 1556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:39:35.478478 1556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:39:35.485066 1556 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:39265
I20260928 14:39:35.485658 1556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:39:35.488274 1689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:39265 every 8 connection(s)
I20260928 14:39:35.494872 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 1556
I20260928 14:39:35.494993 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:39:35.495195 22581 external_mini_cluster.cc:1428] Setting key b470fc6369c57c6196538df4882e2c31
Sep 28 14:39:35 dist-test-slave-wwwb krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606375, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:39:35.504297 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.489237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:39371 (local address 127.22.13.126:40829)
0928 14:39:35.489288 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:39:35.489290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:39:35.489665 (+ 375us) server_negotiation.cc:407] Connection header received
0928 14:39:35.490435 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:39:35.490436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:39:35.490452 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:39:35.490498 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:39:35.491412 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.492233 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.494006 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.494326 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.494371 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.498990 (+ 4619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:39:35.498991 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:39:35.498992 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:39:35.499001 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:39:35.501379 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:39:35.502189 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:39:35.502191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:39:35.502191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:39:35.502307 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:39:35.502757 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:39:35.502758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:39:35.502759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:39:35.502967 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:39:35.503050 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 14:39:35.504081 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0928 14:39:35.504193 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:39:35.505756 1692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.488931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:40829 (local address 127.22.13.66:39371)
0928 14:39:35.489523 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0928 14:39:35.489572 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 14:39:35.490298 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:39:35.490715 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:39:35.490720 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:39:35.490982 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:39:35.491271 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:39:35.491278 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.492362 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.492362 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:39:35.493873 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:39:35.493876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.494481 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.494482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:39:35.494635 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.497032 (+ 2397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:39:35.497045 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:39:35.498796 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:39:35.501556 (+ 2760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:39:35.501567 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:39:35.501571 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:39:35.502068 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:39:35.502407 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:39:35.502407 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:39:35.502408 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:39:35.502649 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:39:35.503106 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:39:35.503114 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:39:35.503888 (+ 774us) client_negotiation.cc:769] Sending connection context
0928 14:39:35.505254 (+ 1366us) client_negotiation.cc:240] Negotiation successful
0928 14:39:35.505484 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":93,"threads_started":1}
I20260928 14:39:35.506620 1690 heartbeater.cc:347] Connected to a master server at 127.22.13.126:40829
I20260928 14:39:35.507174 1690 heartbeater.cc:464] Registering TS with master...
I20260928 14:39:35.507426 1690 heartbeater.cc:511] Master 127.22.13.126:40829 requested a full tablet report, sending...
I20260928 14:39:35.507916 1338 ts_manager.cc:194] Registered new tserver with Master: 34d0603c30fb4539830e3424c89fde7f (127.22.13.66:39265)
I20260928 14:39:35.508415 1338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:39371
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:39:35.538928 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.13.84:51315/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40829
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:42567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:39:35.664218 1695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:39:35.664589 1695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:39:35.664675 1695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:39:35.666646 1695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:39:35.666930 1695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:39:35.667032 1695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:39:35.669387 1695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:42567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.13.84:51315/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:40829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:39:35.669916 1695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:39:35.670274 1695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:39:35.676952 1701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:39:35.677508 1700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:39:35.678083 1703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:39:35.679039 1695 server_base.cc:1034] running on GCE node
I20260928 14:39:35.679286 1695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:39:35.679589 1695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:39:35.680778 1695 hybrid_clock.cc:648] HybridClock initialized: now 1790606375680752 us; error 41 us; skew 500 ppm
Sep 28 14:39:35 dist-test-slave-wwwb krb5kdc[858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606375, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:39:35.684305 1695 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:39:35.684899 1695 webserver.cc:460] Webserver started at http://127.22.13.67:40469/ using document root <none> and password file <none>
I20260928 14:39:35.685173 1695 fs_manager.cc:359] Metadata directory not provided
I20260928 14:39:35.685226 1695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:39:35.685359 1695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:39:35 dist-test-slave-wwwb krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606375, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:39:35.704319 1695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "dd47972297c24422b17181910f2fcde5"
format_stamp: "Formatted at 2026-09-28 14:39:35 on dist-test-slave-wwwb"
server_key: "ab2a1c1b151d601962eaa0f6fa5d7f05"
server_key_iv: "80e32ff086d00c9cd3eccb533d06b963"
server_key_version: "kuduclusterkey@0"
I20260928 14:39:35.704946 1695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "dd47972297c24422b17181910f2fcde5"
format_stamp: "Formatted at 2026-09-28 14:39:35 on dist-test-slave-wwwb"
server_key: "ab2a1c1b151d601962eaa0f6fa5d7f05"
server_key_iv: "80e32ff086d00c9cd3eccb533d06b963"
server_key_version: "kuduclusterkey@0"
I20260928 14:39:35.708006 1695 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260928 14:39:35.726115 1710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:39:35.726387 1695 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 14:39:35.726480 1695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "dd47972297c24422b17181910f2fcde5"
format_stamp: "Formatted at 2026-09-28 14:39:35 on dist-test-slave-wwwb"
server_key: "ab2a1c1b151d601962eaa0f6fa5d7f05"
server_key_iv: "80e32ff086d00c9cd3eccb533d06b963"
server_key_version: "kuduclusterkey@0"
I20260928 14:39:35.726576 1695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:39:35.787590 1695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:39:35.791085 1695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:39:35.791322 1695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:39:35.791606 1695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:39:35.792054 1695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:39:35.792150 1695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:39:35.792224 1695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:39:35.792284 1695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:39:35.799273 1695 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:38081
I20260928 14:39:35.799801 1695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:39:35.800711 1823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:38081 every 8 connection(s)
Sep 28 14:39:35 dist-test-slave-wwwb krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606375, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:39:35.808715 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 1695
I20260928 14:39:35.808821 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:39:35.809007 22581 external_mini_cluster.cc:1428] Setting key 810036313f374a3348c08adcd077552f
I20260928 14:39:35.812317 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.801763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:60815 (local address 127.22.13.126:40829)
0928 14:39:35.801809 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:39:35.801812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:39:35.802340 (+ 528us) server_negotiation.cc:407] Connection header received
0928 14:39:35.803181 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:39:35.803183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:39:35.803195 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:39:35.803228 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:39:35.804103 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.804844 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.806433 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.806654 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.806753 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.809206 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:39:35.809207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:39:35.809208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:39:35.809216 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:39:35.810825 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:39:35.811284 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:39:35.811285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:39:35.811285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:39:35.811349 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:39:35.811630 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:39:35.811631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:39:35.811631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:39:35.811757 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:39:35.811827 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 14:39:35.812139 (+ 312us) server_negotiation.cc:299] Negotiation successful
0928 14:39:35.812223 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:39:35.812551 1826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.801478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:40829 (local address 127.22.13.67:60815)
0928 14:39:35.802174 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0928 14:39:35.802231 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0928 14:39:35.803045 (+ 814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:39:35.803423 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:39:35.803427 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:39:35.803682 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:39:35.803985 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:39:35.803991 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.804980 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.804981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:39:35.806329 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:39:35.806332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.806763 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.806763 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:39:35.806915 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.807513 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:39:35.807524 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:39:35.809079 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:39:35.810945 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:39:35.810953 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:39:35.810956 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:39:35.811204 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:39:35.811420 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:39:35.811421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:39:35.811421 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:39:35.811554 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:39:35.811832 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:39:35.811840 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:39:35.811995 (+ 155us) client_negotiation.cc:769] Sending connection context
0928 14:39:35.812105 (+ 110us) client_negotiation.cc:240] Negotiation successful
0928 14:39:35.812272 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":77,"threads_started":1}
I20260928 14:39:35.813124 1824 heartbeater.cc:347] Connected to a master server at 127.22.13.126:40829
I20260928 14:39:35.813263 1824 heartbeater.cc:464] Registering TS with master...
I20260928 14:39:35.813504 1824 heartbeater.cc:511] Master 127.22.13.126:40829 requested a full tablet report, sending...
I20260928 14:39:35.813922 1338 ts_manager.cc:194] Registered new tserver with Master: dd47972297c24422b17181910f2fcde5 (127.22.13.67:38081)
I20260928 14:39:35.814462 1338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:60815
I20260928 14:39:35.820925 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:39:35.838977 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.826315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42234 (local address 127.22.13.126:40829)
0928 14:39:35.826376 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 14:39:35.826378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:39:35.826690 (+ 312us) server_negotiation.cc:407] Connection header received
0928 14:39:35.826804 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:39:35.826806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:39:35.826819 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:39:35.826851 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:39:35.827733 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.828492 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.829990 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.830300 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.830360 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.832797 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:39:35.832798 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:39:35.832799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:39:35.832808 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:39:35.835313 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:39:35.836693 (+ 1380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:39:35.836695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:39:35.836695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:39:35.836780 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:39:35.837041 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:39:35.837042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:39:35.837042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:39:35.837190 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:39:35.837270 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 14:39:35.838754 (+ 1484us) server_negotiation.cc:299] Negotiation successful
0928 14:39:35.838862 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260928 14:39:35.841642 1338 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:39:35.844075 1338 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:39:35.879886 1841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.860505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:42875 (local address 127.0.0.1:44630)
0928 14:39:35.863658 (+ 3153us) negotiation.cc:107] Waiting for socket to connect
0928 14:39:35.863720 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0928 14:39:35.864851 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:39:35.865199 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:39:35.865202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:39:35.865205 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:39:35.865299 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:39:35.865301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.868815 (+ 3514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.868816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:39:35.870160 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:39:35.870164 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.879531 (+ 9367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.879533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:39:35.879648 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.879665 (+ 17us) client_negotiation.cc:769] Sending connection context
0928 14:39:35.879729 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 14:39:35.879790 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3116,"thread_start_us":97,"threads_started":1}
I20260928 14:39:35.880275 1838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.860603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44630 (local address 127.22.13.65:42875)
0928 14:39:35.863579 (+ 2976us) server_negotiation.cc:206] Beginning negotiation
0928 14:39:35.863584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:39:35.864961 (+ 1377us) server_negotiation.cc:407] Connection header received
0928 14:39:35.865008 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:39:35.865012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:39:35.865030 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:39:35.865099 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:39:35.867487 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.868709 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.870814 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.871598 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.871680 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.871746 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 14:39:35.880137 (+ 8391us) server_negotiation.cc:299] Negotiation successful
0928 14:39:35.880205 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2941,"thread_start_us":70,"threads_started":1}
I20260928 14:39:35.880865 1839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.860928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:38081 (local address 127.0.0.1:55344)
0928 14:39:35.870448 (+ 9520us) negotiation.cc:107] Waiting for socket to connect
0928 14:39:35.870457 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:39:35.870504 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:39:35.871851 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:39:35.871852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:39:35.871855 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:39:35.871908 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:39:35.871909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.874765 (+ 2856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.874766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:39:35.875890 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:39:35.875893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.880620 (+ 4727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.880621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:39:35.880691 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.880696 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:39:35.880750 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 14:39:35.880797 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9507,"thread_start_us":72,"threads_started":1}
I20260928 14:39:35.881052 1484 tablet_service.cc:1467] Processing CreateTablet for tablet 266f9dd63fc34c1a9a01c0098a0fd1f0 (DEFAULT_TABLE table=test-table [id=22906e7d0d624535ae77e7d172268926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:39:35.881327 1842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.861031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55344 (local address 127.22.13.67:38081)
0928 14:39:35.867346 (+ 6315us) server_negotiation.cc:206] Beginning negotiation
0928 14:39:35.867352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:39:35.870569 (+ 3217us) server_negotiation.cc:407] Connection header received
0928 14:39:35.870621 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:39:35.870624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:39:35.870642 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:39:35.870715 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:39:35.872004 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.873009 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.878520 (+ 5511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.879327 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.879409 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.879486 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 14:39:35.881135 (+ 1649us) server_negotiation.cc:299] Negotiation successful
0928 14:39:35.881189 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6260,"thread_start_us":83,"threads_started":1}
I20260928 14:39:35.881474 1484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 266f9dd63fc34c1a9a01c0098a0fd1f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:39:35.883100 1840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.860864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58614 (local address 127.22.13.66:39265)
0928 14:39:35.863494 (+ 2630us) server_negotiation.cc:206] Beginning negotiation
0928 14:39:35.863500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:39:35.870397 (+ 6897us) server_negotiation.cc:407] Connection header received
0928 14:39:35.873184 (+ 2787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:39:35.873189 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:39:35.873210 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:39:35.873271 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:39:35.876040 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.877142 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.881635 (+ 4493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.882490 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.882545 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.882621 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:39:35.882972 (+ 351us) server_negotiation.cc:299] Negotiation successful
0928 14:39:35.883020 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2577,"thread_start_us":73,"threads_started":1}
I20260928 14:39:35.884254 1758 tablet_service.cc:1467] Processing CreateTablet for tablet 266f9dd63fc34c1a9a01c0098a0fd1f0 (DEFAULT_TABLE table=test-table [id=22906e7d0d624535ae77e7d172268926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:39:35.884625 1758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 266f9dd63fc34c1a9a01c0098a0fd1f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:39:35.883488 1837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.860757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:39265 (local address 127.0.0.1:58614)
0928 14:39:35.870325 (+ 9568us) negotiation.cc:107] Waiting for socket to connect
0928 14:39:35.870338 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:39:35.871779 (+ 1441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:39:35.873391 (+ 1612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:39:35.873392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:39:35.873396 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:39:35.874650 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:39:35.874656 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.877259 (+ 2603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.877260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:39:35.878383 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:39:35.878387 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.882688 (+ 4301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.882689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:39:35.882761 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.882764 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:39:35.883360 (+ 596us) client_negotiation.cc:240] Negotiation successful
0928 14:39:35.883417 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9540,"thread_start_us":74,"threads_started":1}
I20260928 14:39:35.888810 1844 tablet_bootstrap.cc:492] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5: Bootstrap starting.
I20260928 14:39:35.889107 1620 tablet_service.cc:1467] Processing CreateTablet for tablet 266f9dd63fc34c1a9a01c0098a0fd1f0 (DEFAULT_TABLE table=test-table [id=22906e7d0d624535ae77e7d172268926]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:39:35.889518 1620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 266f9dd63fc34c1a9a01c0098a0fd1f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:39:35.890080 1844 tablet_bootstrap.cc:654] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5: Neither blocks nor log segments found. Creating new log.
I20260928 14:39:35.890477 1844 log.cc:824] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5: Log is configured to *not* fsync() on all Append() calls
I20260928 14:39:35.893241 1846 tablet_bootstrap.cc:492] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f: Bootstrap starting.
I20260928 14:39:35.893604 1843 tablet_bootstrap.cc:492] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 74b381a4f2664b7686636fcd6bac1046: Bootstrap starting.
I20260928 14:39:35.894384 1846 tablet_bootstrap.cc:654] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f: Neither blocks nor log segments found. Creating new log.
I20260928 14:39:35.894402 1843 tablet_bootstrap.cc:654] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 74b381a4f2664b7686636fcd6bac1046: Neither blocks nor log segments found. Creating new log.
I20260928 14:39:35.894726 1846 log.cc:824] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f: Log is configured to *not* fsync() on all Append() calls
I20260928 14:39:35.894974 1843 log.cc:824] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 74b381a4f2664b7686636fcd6bac1046: Log is configured to *not* fsync() on all Append() calls
I20260928 14:39:35.895802 1844 tablet_bootstrap.cc:492] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5: No bootstrap required, opened a new log
I20260928 14:39:35.895874 1844 ts_tablet_manager.cc:1397] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20260928 14:39:35.897794 1844 raft_consensus.cc:348] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b381a4f2664b7686636fcd6bac1046" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42875 } } peers { permanent_uuid: "34d0603c30fb4539830e3424c89fde7f" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39265 } } peers { permanent_uuid: "dd47972297c24422b17181910f2fcde5" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38081 } }
I20260928 14:39:35.898105 1844 raft_consensus.cc:374] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:39:35.898145 1844 raft_consensus.cc:729] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd47972297c24422b17181910f2fcde5, State: Initialized, Role: FOLLOWER
I20260928 14:39:35.898260 1844 consensus_queue.cc:260] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b381a4f2664b7686636fcd6bac1046" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42875 } } peers { permanent_uuid: "34d0603c30fb4539830e3424c89fde7f" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39265 } } peers { permanent_uuid: "dd47972297c24422b17181910f2fcde5" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38081 } }
I20260928 14:39:35.898518 1844 ts_tablet_manager.cc:1428] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260928 14:39:35.898851 1824 heartbeater.cc:503] Master 127.22.13.126:40829 was elected leader, sending a full tablet report...
I20260928 14:39:35.901213 1843 tablet_bootstrap.cc:492] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 74b381a4f2664b7686636fcd6bac1046: No bootstrap required, opened a new log
I20260928 14:39:35.901292 1843 ts_tablet_manager.cc:1397] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 74b381a4f2664b7686636fcd6bac1046: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260928 14:39:35.903923 1843 raft_consensus.cc:348] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 74b381a4f2664b7686636fcd6bac1046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b381a4f2664b7686636fcd6bac1046" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42875 } } peers { permanent_uuid: "34d0603c30fb4539830e3424c89fde7f" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39265 } } peers { permanent_uuid: "dd47972297c24422b17181910f2fcde5" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38081 } }
I20260928 14:39:35.904134 1843 raft_consensus.cc:374] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 74b381a4f2664b7686636fcd6bac1046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:39:35.904165 1843 raft_consensus.cc:729] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 74b381a4f2664b7686636fcd6bac1046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b381a4f2664b7686636fcd6bac1046, State: Initialized, Role: FOLLOWER
I20260928 14:39:35.904474 1843 consensus_queue.cc:260] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 74b381a4f2664b7686636fcd6bac1046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b381a4f2664b7686636fcd6bac1046" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42875 } } peers { permanent_uuid: "34d0603c30fb4539830e3424c89fde7f" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39265 } } peers { permanent_uuid: "dd47972297c24422b17181910f2fcde5" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38081 } }
I20260928 14:39:35.904834 1843 ts_tablet_manager.cc:1428] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 74b381a4f2664b7686636fcd6bac1046: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260928 14:39:35.905011 1550 heartbeater.cc:503] Master 127.22.13.126:40829 was elected leader, sending a full tablet report...
I20260928 14:39:35.905519 1846 tablet_bootstrap.cc:492] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f: No bootstrap required, opened a new log
I20260928 14:39:35.905668 1846 ts_tablet_manager.cc:1397] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260928 14:39:35.907732 1846 raft_consensus.cc:348] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b381a4f2664b7686636fcd6bac1046" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42875 } } peers { permanent_uuid: "34d0603c30fb4539830e3424c89fde7f" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39265 } } peers { permanent_uuid: "dd47972297c24422b17181910f2fcde5" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38081 } }
I20260928 14:39:35.908039 1846 raft_consensus.cc:374] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:39:35.908123 1846 raft_consensus.cc:729] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34d0603c30fb4539830e3424c89fde7f, State: Initialized, Role: FOLLOWER
I20260928 14:39:35.908324 1846 consensus_queue.cc:260] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b381a4f2664b7686636fcd6bac1046" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42875 } } peers { permanent_uuid: "34d0603c30fb4539830e3424c89fde7f" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39265 } } peers { permanent_uuid: "dd47972297c24422b17181910f2fcde5" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38081 } }
I20260928 14:39:35.908685 1846 ts_tablet_manager.cc:1428] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:39:35.908849 1690 heartbeater.cc:503] Master 127.22.13.126:40829 was elected leader, sending a full tablet report...
I20260928 14:39:35.913996 1849 raft_consensus.cc:482] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:39:35.914158 1849 raft_consensus.cc:504] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b381a4f2664b7686636fcd6bac1046" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42875 } } peers { permanent_uuid: "34d0603c30fb4539830e3424c89fde7f" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39265 } } peers { permanent_uuid: "dd47972297c24422b17181910f2fcde5" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38081 } }
I20260928 14:39:35.914567 1849 leader_election.cc:288] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74b381a4f2664b7686636fcd6bac1046 (127.22.13.65:42875), 34d0603c30fb4539830e3424c89fde7f (127.22.13.66:39265)
I20260928 14:39:35.919119 1826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.914758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:42875 (local address 127.22.13.67:58323)
0928 14:39:35.915016 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0928 14:39:35.915027 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:39:35.915080 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:39:35.915282 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:39:35.915283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:39:35.915285 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:39:35.915357 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:39:35.915359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.916354 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.916356 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:39:35.917564 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:39:35.917568 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.918845 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.918846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:39:35.918942 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.918946 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:39:35.918991 (+ 45us) client_negotiation.cc:240] Negotiation successful
0928 14:39:35.919037 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239}
I20260928 14:39:35.919852 1838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.914855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:58323 (local address 127.22.13.65:42875)
0928 14:39:35.914955 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0928 14:39:35.914957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:39:35.915083 (+ 126us) server_negotiation.cc:407] Connection header received
0928 14:39:35.915138 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:39:35.915139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:39:35.915152 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:39:35.915221 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:39:35.915433 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.916216 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.918111 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.918724 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.918799 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.918809 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 14:39:35.919734 (+ 925us) server_negotiation.cc:299] Negotiation successful
0928 14:39:35.919786 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 14:39:35.920127 1504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "266f9dd63fc34c1a9a01c0098a0fd1f0" candidate_uuid: "dd47972297c24422b17181910f2fcde5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74b381a4f2664b7686636fcd6bac1046" is_pre_election: true
I20260928 14:39:35.920279 1504 raft_consensus.cc:2454] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 74b381a4f2664b7686636fcd6bac1046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd47972297c24422b17181910f2fcde5 in term 0.
I20260928 14:39:35.920627 1711 leader_election.cc:302] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74b381a4f2664b7686636fcd6bac1046, dd47972297c24422b17181910f2fcde5; no voters:
I20260928 14:39:35.920789 1849 raft_consensus.cc:2781] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:39:35.920830 1849 raft_consensus.cc:482] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:39:35.920849 1849 raft_consensus.cc:3037] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:39:35.921797 1849 raft_consensus.cc:504] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b381a4f2664b7686636fcd6bac1046" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42875 } } peers { permanent_uuid: "34d0603c30fb4539830e3424c89fde7f" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39265 } } peers { permanent_uuid: "dd47972297c24422b17181910f2fcde5" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38081 } }
I20260928 14:39:35.921928 1849 leader_election.cc:288] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [CANDIDATE]: Term 1 election: Requested vote from peers 74b381a4f2664b7686636fcd6bac1046 (127.22.13.65:42875), 34d0603c30fb4539830e3424c89fde7f (127.22.13.66:39265)
I20260928 14:39:35.922366 1504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "266f9dd63fc34c1a9a01c0098a0fd1f0" candidate_uuid: "dd47972297c24422b17181910f2fcde5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74b381a4f2664b7686636fcd6bac1046"
I20260928 14:39:35.922441 1504 raft_consensus.cc:3037] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 74b381a4f2664b7686636fcd6bac1046 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:39:35.923329 1504 raft_consensus.cc:2454] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 74b381a4f2664b7686636fcd6bac1046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd47972297c24422b17181910f2fcde5 in term 1.
I20260928 14:39:35.923673 1711 leader_election.cc:302] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74b381a4f2664b7686636fcd6bac1046, dd47972297c24422b17181910f2fcde5; no voters:
I20260928 14:39:35.923796 1849 raft_consensus.cc:2781] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:39:35.923959 1849 raft_consensus.cc:686] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [term 1 LEADER]: Becoming Leader. State: Replica: dd47972297c24422b17181910f2fcde5, State: Running, Role: LEADER
I20260928 14:39:35.924050 1849 consensus_queue.cc:237] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b381a4f2664b7686636fcd6bac1046" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42875 } } peers { permanent_uuid: "34d0603c30fb4539830e3424c89fde7f" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39265 } } peers { permanent_uuid: "dd47972297c24422b17181910f2fcde5" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38081 } }
I20260928 14:39:35.924892 1337 catalog_manager.cc:5495] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd47972297c24422b17181910f2fcde5 (127.22.13.67). New cstate: current_term: 1 leader_uuid: "dd47972297c24422b17181910f2fcde5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74b381a4f2664b7686636fcd6bac1046" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34d0603c30fb4539830e3424c89fde7f" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd47972297c24422b17181910f2fcde5" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38081 } health_report { overall_health: HEALTHY } } }
I20260928 14:39:35.930151 1826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.914864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:39265 (local address 127.22.13.67:55127)
0928 14:39:35.919212 (+ 4348us) negotiation.cc:107] Waiting for socket to connect
0928 14:39:35.919237 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0928 14:39:35.919279 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:39:35.926043 (+ 6764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:39:35.926044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:39:35.926046 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:39:35.926095 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:39:35.926096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.926992 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.926993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:39:35.927985 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:39:35.927989 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.929863 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.929864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:39:35.929962 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.929967 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:39:35.930025 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 14:39:35.930071 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4336,"thread_start_us":109,"threads_started":1}
I20260928 14:39:35.930568 1840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.915009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:55127 (local address 127.22.13.66:39265)
0928 14:39:35.915048 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 14:39:35.915050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:39:35.919352 (+ 4302us) server_negotiation.cc:407] Connection header received
0928 14:39:35.919382 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:39:35.919383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:39:35.919393 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:39:35.919436 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:39:35.926184 (+ 6748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.926866 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.928092 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.928650 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.928721 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.928730 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:39:35.930420 (+ 1690us) server_negotiation.cc:299] Negotiation successful
0928 14:39:35.930486 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:39:35.931169 1640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "266f9dd63fc34c1a9a01c0098a0fd1f0" candidate_uuid: "dd47972297c24422b17181910f2fcde5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34d0603c30fb4539830e3424c89fde7f" is_pre_election: true
I20260928 14:39:35.931330 1640 raft_consensus.cc:2454] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd47972297c24422b17181910f2fcde5 in term 0.
I20260928 14:39:35.931432 1637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "266f9dd63fc34c1a9a01c0098a0fd1f0" candidate_uuid: "dd47972297c24422b17181910f2fcde5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34d0603c30fb4539830e3424c89fde7f"
I20260928 14:39:35.931485 1637 raft_consensus.cc:3037] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:39:35.932384 1637 raft_consensus.cc:2454] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd47972297c24422b17181910f2fcde5 in term 1.
I20260928 14:39:35.938531 1842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:35.935091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55348 (local address 127.22.13.67:38081)
0928 14:39:35.935137 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:39:35.935139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:39:35.935152 (+ 13us) server_negotiation.cc:407] Connection header received
0928 14:39:35.935193 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:39:35.935194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:39:35.935216 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:39:35.935313 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:39:35.935499 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.936202 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.937318 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:39:35.937645 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:39:35.937704 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:39:35.937910 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:39:35.938077 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:39:35.938151 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:39:35.938399 (+ 248us) server_negotiation.cc:299] Negotiation successful
0928 14:39:35.938421 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:39:35.951937 1637 raft_consensus.cc:1261] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f [term 1 FOLLOWER]: Refusing update from remote peer dd47972297c24422b17181910f2fcde5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:39:35.954373 1853 consensus_queue.cc:1035] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34d0603c30fb4539830e3424c89fde7f" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:39:35.958344 1504 raft_consensus.cc:1261] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 74b381a4f2664b7686636fcd6bac1046 [term 1 FOLLOWER]: Refusing update from remote peer dd47972297c24422b17181910f2fcde5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:39:35.958778 1853 consensus_queue.cc:1035] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74b381a4f2664b7686636fcd6bac1046" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:39:35.963424 1856 mvcc.cc:205] Tried to move back new op lower bound from 7334323715895865344 to 7334323715785486336. Current Snapshot: MvccSnapshot[applied={T|T < 7334323715895865344}]
I20260928 14:39:35.966622 1337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:39:35.966732 1337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:39:35.967711 1337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8523c484769a4930bbfa89054c5a626b: Sending DeleteTablet for 3 replicas of tablet 266f9dd63fc34c1a9a01c0098a0fd1f0
I20260928 14:39:35.968140 1484 tablet_service.cc:1514] Processing DeleteTablet for tablet 266f9dd63fc34c1a9a01c0098a0fd1f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:39:35 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:44630
I20260928 14:39:35.968499 1620 tablet_service.cc:1514] Processing DeleteTablet for tablet 266f9dd63fc34c1a9a01c0098a0fd1f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:39:35 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:58614
I20260928 14:39:35.969085 1758 tablet_service.cc:1514] Processing DeleteTablet for tablet 266f9dd63fc34c1a9a01c0098a0fd1f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:39:35 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:55344
I20260928 14:39:35.969635 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 1419
I20260928 14:39:35.974195 1865 tablet_replica.cc:330] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f: stopping tablet replica
I20260928 14:39:35.974377 1866 tablet_replica.cc:330] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5: stopping tablet replica
I20260928 14:39:35.974586 1866 raft_consensus.cc:2229] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:39:35.974748 1865 raft_consensus.cc:2229] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:39:35.974912 1866 raft_consensus.cc:2258] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:39:35.975033 1865 raft_consensus.cc:2258] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:39:35.975705 1866 ts_tablet_manager.cc:1905] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:39:35.981099 1866 ts_tablet_manager.cc:1918] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:39:35.981292 1866 log.cc:1196] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/266f9dd63fc34c1a9a01c0098a0fd1f0
I20260928 14:39:35.981734 1866 ts_tablet_manager.cc:1939] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P dd47972297c24422b17181910f2fcde5: Deleting consensus metadata
I20260928 14:39:35.982376 1321 catalog_manager.cc:4841] TS dd47972297c24422b17181910f2fcde5 (127.22.13.67:38081): tablet 266f9dd63fc34c1a9a01c0098a0fd1f0 (table test-table [id=22906e7d0d624535ae77e7d172268926]) successfully deleted
I20260928 14:39:35.984527 1865 ts_tablet_manager.cc:1905] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:39:35.986793 1865 ts_tablet_manager.cc:1918] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:39:35.986898 1865 log.cc:1196] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/266f9dd63fc34c1a9a01c0098a0fd1f0
I20260928 14:39:35.987201 1865 ts_tablet_manager.cc:1939] T 266f9dd63fc34c1a9a01c0098a0fd1f0 P 34d0603c30fb4539830e3424c89fde7f: Deleting consensus metadata
I20260928 14:39:35.987731 1321 catalog_manager.cc:4841] TS 34d0603c30fb4539830e3424c89fde7f (127.22.13.66:39265): tablet 266f9dd63fc34c1a9a01c0098a0fd1f0 (table test-table [id=22906e7d0d624535ae77e7d172268926]) successfully deleted
W20260928 14:39:36.002322 1711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:42875 (error 108)
W20260928 14:39:36.002933 1321 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:42875 (error 108)
I20260928 14:39:36.009382 1826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:36.009193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:42875 (local address 127.22.13.67:36187)
0928 14:39:36.009266 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0928 14:39:36.009318 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:42875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260928 14:39:36.009887 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 1556
I20260928 14:39:36.010915 1837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:39:36.010758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:42875 (local address 127.0.0.1:44646)
0928 14:39:36.010812 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 14:39:36.010870 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:42875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260928 14:39:36.011072 1321 catalog_manager.cc:4584] TS 74b381a4f2664b7686636fcd6bac1046 (127.22.13.65:42875): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 266f9dd63fc34c1a9a01c0098a0fd1f0: Network error: Client connection negotiation failed: client connection to 127.22.13.65:42875: connect: Connection refused (error 111)
I20260928 14:39:36.017562 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 1695
Sep 28 14:39:36 dist-test-slave-wwwb krb5kdc[858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606375, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:39:36.022799 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 1305
I20260928 14:39:36.029575 22581 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260928 14:39:36.374989 22581 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260928 14:39:36.375092 22581 mini_ranger.cc:66] Stopping Ranger...
I20260928 14:39:36.721299 22581 mini_ranger.cc:68] Stopped Ranger
2026-09-28 14:39:36.721 UTC [888] LOG: received smart shutdown request
2026-09-28 14:39:36.724 UTC [888] LOG: background worker "logical replication launcher" (PID 897) exited with exit code 1
2026-09-28 14:39:36.724 UTC [892] LOG: shutting down
2026-09-28 14:39:36.866 UTC [888] LOG: database system is shut down
2026-09-28T14:39:36Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48673 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:39:37 dist-test-slave-wwwb krb5kdc[1877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:39:37 dist-test-slave-wwwb krb5kdc[1877](info): set up 2 sockets
Sep 28 14:39:37 dist-test-slave-wwwb krb5kdc[1877](info): commencing operation
krb5kdc: starting...
W20260928 14:39:38.014127 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:39:38 dist-test-slave-wwwb krb5kdc[1877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:39:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:39:38Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.13.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.13.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.13.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/rangeradmin_127.22.13.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.13.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/rangeradmin_127.22.13.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.13.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.13.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.13.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/rangerlookup_127.22.13.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.13.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/rangerlookup_127.22.13.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.13.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.13.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.13.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/HTTP_127.22.13.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.13.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/HTTP_127.22.13.84@KRBTEST.COM.keytab.
I20260928 14:39:38.181800 22581 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskT1ZEJk/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/postgres -l logfile start
2026-09-28 14:39:40.549 UTC [1907] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-28 14:39:40.549 UTC [1907] LOG: listening on IPv4 address "127.22.13.84", port 46511
2026-09-28 14:39:40.553 UTC [1907] LOG: listening on Unix socket "/tmp/.s.PGSQL.46511"
2026-09-28 14:39:40.573 UTC [1910] LOG: database system was shut down at 2026-09-28 14:39:39 UTC
2026-09-28 14:39:40.577 UTC [1907] LOG: database system is ready to accept connections
I20260928 14:39:41.546113 22581 mini_postgres.cc:96] Postgres bound to 46511
2026-09-28 14:39:41.552 UTC [1918] FATAL: database "slave" does not exist
127.22.13.84:46511 - accepting connections
I20260928 14:39:41.552949 22581 mini_ranger.cc:161] Starting Ranger...
I20260928 14:39:41.568660 22581 mini_ranger.cc:84] Created miniranger Postgres user
I20260928 14:39:41.991689 22581 mini_ranger.cc:87] Created ranger Postgres database
I20260928 14:39:41.991787 22581 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-admin
2026-09-28 14:39:42,275 [I] DB FLAVOR :POSTGRES
2026-09-28 14:39:42,276 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 14:39:42,276 [I] Checking connection..
2026-09-28 14:39:42,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-28 14:39:42,556 [I] Checking connection passed.
2026-09-28 14:39:42,556 [I] --------- Verifying version history table ---------
2026-09-28 14:39:42,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:39:42,813 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 14:39:42,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:39:43,095 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 14:39:43,095 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-28 14:39:43,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-28 14:39:43.341 UTC [1982] WARNING: there is no transaction in progress
2026-09-28 14:39:43,347 [I] create_dbversion_catalog.sql file imported successfully
2026-09-28 14:39:43,348 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:39:43,622 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-28 14:39:43,622 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 14:39:43,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-28 14:39:43,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-28 14:39:44,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-wwwb.c.gcp-upstream.internal','N') ;"
2026-09-28 14:39:44,412 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 14:39:44,663 [I] Table x_portal_user does not exist in database ranger
2026-09-28 14:39:44,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 14:39:44,939 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-28 14:39:44,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 14:39:45,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 14:39:45,453 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-28 14:39:45,453 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-28 14:39:46.467 UTC [2090] WARNING: there is no transaction in progress
2026-09-28 14:39:46.483 UTC [2090] WARNING: there is no transaction in progress
2026-09-28 14:39:46.500 UTC [2090] WARNING: there is no transaction in progress
2026-09-28 14:39:46.516 UTC [2090] WARNING: there is no transaction in progress
2026-09-28 14:39:46.533 UTC [2090] WARNING: there is no transaction in progress
2026-09-28 14:39:46.551 UTC [2090] WARNING: there is no transaction in progress
2026-09-28 14:39:46.617 UTC [2090] WARNING: there is no transaction in progress
2026-09-28 14:39:46.624 UTC [2090] WARNING: there is no transaction in progress
2026-09-28 14:39:46.638 UTC [2090] WARNING: there is no transaction in progress
2026-09-28 14:39:46.648 UTC [2090] WARNING: there is no transaction in progress
2026-09-28 14:39:46.658 UTC [2090] WARNING: there is no transaction in progress
2026-09-28 14:39:46.672 UTC [2090] WARNING: there is no transaction in progress
2026-09-28 14:39:46.682 UTC [2090] WARNING: there is no transaction in progress
2026-09-28 14:39:46.688 UTC [2090] WARNING: there is no transaction in progress
2026-09-28 14:39:46.695 UTC [2090] WARNING: there is no transaction in progress
2026-09-28 14:39:46.701 UTC [2090] WARNING: there is no transaction in progress
2026-09-28 14:39:47,314 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-28 14:39:47,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-28 14:39:47,570 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-28 14:39:47,570 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 14:39:47,851 [I] Table x_portal_user already exists in database 'ranger'
2026-09-28 14:39:47,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 14:39:48,123 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-28 14:39:48,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 14:39:48,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 14:39:48,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-wwwb.c.gcp-upstream.internal';"
2026-09-28 14:39:48,890 [I] CORE_DB_SCHEMA import status has been updated
2026-09-28 14:39:48,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-28 14:39:49,165 [I] DB_PATCHES have already been applied
I20260928 14:39:49.171066 22581 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-admin:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskT1ZEJk/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260928 14:39:49.171171 22581 mini_ranger.cc:193] Using host: 127.22.13.84
I20260928 14:39:49.174392 22581 mini_ranger.cc:227] Ranger admin URL: http://127.22.13.84:48415
Sep 28, 2026 2:39:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 2:39:49 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 2:39:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-admin/ews/webapp
Sep 28, 2026 2:39:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 28, 2026 2:39:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 28, 2026 2:39:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-admin/ews/webapp].
Sep 28, 2026 2:39:50 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260928 14:39:50.298480 22581 mini_ranger.cc:160] Time spent starting Ranger: real 8.746s user 0.001s sys 0.008s
Sep 28, 2026 2:40:11 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 28, 2026 2:40:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 28, 2026 2:40:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 28, 2026 2:40:11 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 28, 2026 2:40:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 28, 2026 2:40:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 28, 2026 2:40:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 28, 2026 2:40:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 28, 2026 2:40:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 28, 2026 2:40:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 28, 2026 2:40:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 28, 2026 2:40:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 28, 2026 2:40:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 28, 2026 2:40:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 28, 2026 2:40:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 28, 2026 2:40:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 28, 2026 2:40:11 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 28, 2026 2:40:12 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260928 14:40:13.366808 22581 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.13.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.13.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.13.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/rangerkms_127.22.13.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.13.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/rangerkms_127.22.13.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.13.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.13.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.13.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/HTTP_127.22.13.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.13.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/HTTP_127.22.13.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 28 14:40:13 dist-test-slave-wwwb krb5kdc[1877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606413, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260928 14:40:13.451627 22581 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260928 14:40:13.461473 22581 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260928 14:40:14.277244 22581 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260928 14:40:14.277345 22581 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-kms
I20260928 14:40:14.277364 22581 mini_ranger_kms.cc:227] Using postgres at 127.22.13.84:46511
2026-09-28 14:40:14,452 [I] DB FLAVOR :POSTGRES
2026-09-28 14:40:14,453 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 14:40:14,453 [I] Checking connection
2026-09-28 14:40:14,453 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-28 14:40:14,709 [I] connection success
2026-09-28 14:40:14,709 [I] --------- Verifying Ranger DB tables ---------
2026-09-28 14:40:14,709 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-28 14:40:14,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-28 14:40:14,956 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-28 14:40:14,956 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 14:40:14,956 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-28 14:40:14,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.13.84:46511/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-28 14:40:15,221 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260928 14:40:15.781701 22581 mini_ranger_kms.cc:310] Created kms service
I20260928 14:40:15.980218 22581 mini_ranger_kms.cc:326] Created kudu user
I20260928 14:40:16.031745 22581 mini_ranger_kms.cc:343] Created rangerkms user
Sep 28 14:40:16 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606413, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:40:16.527052 22581 mini_ranger_kms.cc:384] Added ranger policy
I20260928 14:40:16.527272 22581 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-kms:/tmp/dist-test-taskT1ZEJk/build/release/bin/postgresql.jar:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskT1ZEJk/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260928 14:40:16.527294 22581 mini_ranger_kms.cc:240] Using host: 127.22.13.84
I20260928 14:40:16.531358 22581 mini_ranger_kms.cc:276] Ranger KMS PID: 2271
I20260928 14:40:16.531459 22581 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.13.84:45063
Sep 28, 2026 2:40:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 2:40:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 2:40:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 28, 2026 2:40:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 28, 2026 2:40:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-kms/ews/webapp].
Sep 28, 2026 2:40:18 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260928 14:40:18.825609 22581 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.374s user 0.013s sys 0.145s
I20260928 14:40:18.825716 22581 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260928 14:40:18.825784 22581 mini_ranger_kms.cc:401] 127.22.13.84:45063/kms/v1/keys
Sep 28 14:40:18 dist-test-slave-wwwb krb5kdc[1877](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790606418, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.13.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 28 14:40:19 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606418, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.13.84@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
Sep 28 14:40:19 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606418, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.13.84@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
Sep 28 14:40:22 dist-test-slave-wwwb krb5kdc[1877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:40:22.080968 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:35737
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:35737
--ranger_config_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.13.84:45063/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:22.170157 2321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:22.170440 2321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:22.170467 2321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:22.171814 2321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:40:22.171875 2321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:22.171890 2321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:22.171902 2321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:40:22.171914 2321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:40:22.173671 2321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.13.84:45063/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:35737
--ranger_config_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:35737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:22.173986 2321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:22.174263 2321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:22.179926 2329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:22.179916 2326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:22.180071 2321 server_base.cc:1034] running on GCE node
W20260928 14:40:22.179916 2327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:22.180627 2321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:22.181022 2321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:22.182211 2321 hybrid_clock.cc:648] HybridClock initialized: now 1790606422182183 us; error 44 us; skew 500 ppm
Sep 28 14:40:22 dist-test-slave-wwwb krb5kdc[1877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606422, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:22.192783 2321 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:40:22.193405 2321 webserver.cc:460] Webserver started at http://127.22.13.126:45193/ using document root <none> and password file <none>
I20260928 14:40:22.193661 2321 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:22.193722 2321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:22.193892 2321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:40:22 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606422, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:40:22.243885 2321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "909a182c9df043a5b0c506cbbc4d6c63"
format_stamp: "Formatted at 2026-09-28 14:40:22 on dist-test-slave-wwwb"
server_key: "1652fab200a02b6d3465d4b60ac5d4bc"
server_key_iv: "fa739b89072e6d6135e3c6bec6ac883b"
server_key_version: "kuduclusterkey@0"
I20260928 14:40:22.244369 2321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "909a182c9df043a5b0c506cbbc4d6c63"
format_stamp: "Formatted at 2026-09-28 14:40:22 on dist-test-slave-wwwb"
server_key: "1652fab200a02b6d3465d4b60ac5d4bc"
server_key_iv: "fa739b89072e6d6135e3c6bec6ac883b"
server_key_version: "kuduclusterkey@0"
I20260928 14:40:22.245983 2321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:40:22.258977 2336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:22.259264 2321 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:40:22.259384 2321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "909a182c9df043a5b0c506cbbc4d6c63"
format_stamp: "Formatted at 2026-09-28 14:40:22 on dist-test-slave-wwwb"
server_key: "1652fab200a02b6d3465d4b60ac5d4bc"
server_key_iv: "fa739b89072e6d6135e3c6bec6ac883b"
server_key_version: "kuduclusterkey@0"
I20260928 14:40:22.259475 2321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:22.289398 2321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:22.291666 2321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:22.291837 2321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:22.297225 2321 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:35737
I20260928 14:40:22.297302 2388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:35737 every 8 connection(s)
I20260928 14:40:22.297716 2321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:40:22.298512 2389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:22.300879 2389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63: Bootstrap starting.
I20260928 14:40:22.301503 2389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:22.301813 2389 log.cc:824] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:22.302597 2389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63: No bootstrap required, opened a new log
I20260928 14:40:22.304673 2389 raft_consensus.cc:348] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "909a182c9df043a5b0c506cbbc4d6c63" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35737 } }
I20260928 14:40:22.304862 2389 raft_consensus.cc:374] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:22.304929 2389 raft_consensus.cc:729] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 909a182c9df043a5b0c506cbbc4d6c63, State: Initialized, Role: FOLLOWER
I20260928 14:40:22.305123 2389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "909a182c9df043a5b0c506cbbc4d6c63" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35737 } }
I20260928 14:40:22.305226 2389 raft_consensus.cc:388] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:40:22.305270 2389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:40:22.305320 2389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:22.305926 2389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "909a182c9df043a5b0c506cbbc4d6c63" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35737 } }
I20260928 14:40:22.306104 2389 leader_election.cc:302] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 909a182c9df043a5b0c506cbbc4d6c63; no voters:
I20260928 14:40:22.306305 2389 leader_election.cc:288] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:40:22.306334 2392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:40:22.306449 2392 raft_consensus.cc:686] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 1 LEADER]: Becoming Leader. State: Replica: 909a182c9df043a5b0c506cbbc4d6c63, State: Running, Role: LEADER
I20260928 14:40:22.306593 2389 sys_catalog.cc:564] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:40:22.306608 2392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "909a182c9df043a5b0c506cbbc4d6c63" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35737 } }
I20260928 14:40:22.307065 2394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 909a182c9df043a5b0c506cbbc4d6c63. Latest consensus state: current_term: 1 leader_uuid: "909a182c9df043a5b0c506cbbc4d6c63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "909a182c9df043a5b0c506cbbc4d6c63" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35737 } } }
I20260928 14:40:22.307240 2394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:22.307457 2393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "909a182c9df043a5b0c506cbbc4d6c63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "909a182c9df043a5b0c506cbbc4d6c63" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35737 } } }
I20260928 14:40:22.307571 2393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:22.307741 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 2321
I20260928 14:40:22.307853 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:40:22.308017 22581 external_mini_cluster.cc:1428] Setting key 3c78d0982a8a01471e4ffe9c20effe96
I20260928 14:40:22.308947 2389 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 14:40:22.309252 2401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:40:22.309799 2401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:40:22.314754 2401 catalog_manager.cc:1347] Generated new cluster ID: 6351865a421b40aeb43d831c42ad2e80
I20260928 14:40:22.314832 2401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:40:22 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:22.331691 2409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:22.308663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37856 (local address 127.22.13.126:35737)
0928 14:40:22.309637 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:22.309647 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:22.309679 (+ 32us) server_negotiation.cc:407] Connection header received
0928 14:40:22.309809 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:22.309816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:22.309878 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:22.310064 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:22.312628 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:22.321930 (+ 9302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:22.323314 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:22.324050 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:22.324202 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:22.327566 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:22.327571 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:22.327572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:22.327588 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:22.329752 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:22.330257 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:22.330261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:22.330261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:22.330391 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:22.330602 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:22.330603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:22.330604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:22.330968 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:22.331141 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:22.331255 (+ 114us) server_negotiation.cc:299] Negotiation successful
0928 14:40:22.331400 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":664,"threads_started":1}
I20260928 14:40:22.333151 2401 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:40:22.333724 2401 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:40:22.339336 2401 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63: Generated new TSK 0
I20260928 14:40:22.339568 2401 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:40:23.101409 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.13.84:45063/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35737
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:23.193861 2434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:23.194216 2434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:23.194260 2434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:23.195706 2434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:23.195777 2434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:23.195823 2434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:40:23.197562 2434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.13.84:45063/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:23.197863 2434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:23.198153 2434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:23.203401 2439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:23.203395 2442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:23.203711 2434 server_base.cc:1034] running on GCE node
W20260928 14:40:23.203498 2440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:23.204020 2434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:23.204298 2434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:23.205476 2434 hybrid_clock.cc:648] HybridClock initialized: now 1790606423205455 us; error 45 us; skew 500 ppm
Sep 28 14:40:23 dist-test-slave-wwwb krb5kdc[1877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606423, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:23.208809 2434 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:40:23.209498 2434 webserver.cc:460] Webserver started at http://127.22.13.65:39295/ using document root <none> and password file <none>
I20260928 14:40:23.209802 2434 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:23.209882 2434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:23.210107 2434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:40:23 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606423, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:40:23.236456 2434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "eaf2b4a72dd3425ea0889445dda7b897"
format_stamp: "Formatted at 2026-09-28 14:40:23 on dist-test-slave-wwwb"
server_key: "7928d447f7bb4c9502c2d00ffd912220"
server_key_iv: "25e062b8ffa0007cc7407158a02a9444"
server_key_version: "kuduclusterkey@0"
I20260928 14:40:23.237149 2434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "eaf2b4a72dd3425ea0889445dda7b897"
format_stamp: "Formatted at 2026-09-28 14:40:23 on dist-test-slave-wwwb"
server_key: "7928d447f7bb4c9502c2d00ffd912220"
server_key_iv: "25e062b8ffa0007cc7407158a02a9444"
server_key_version: "kuduclusterkey@0"
I20260928 14:40:23.245906 2434 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.002s
I20260928 14:40:23.260486 2449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:23.260710 2434 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:40:23.260810 2434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "eaf2b4a72dd3425ea0889445dda7b897"
format_stamp: "Formatted at 2026-09-28 14:40:23 on dist-test-slave-wwwb"
server_key: "7928d447f7bb4c9502c2d00ffd912220"
server_key_iv: "25e062b8ffa0007cc7407158a02a9444"
server_key_version: "kuduclusterkey@0"
I20260928 14:40:23.260933 2434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:23.274261 2434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:23.276782 2434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:23.276957 2434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:23.277251 2434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:23.277725 2434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:40:23.277787 2434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:23.277820 2434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:40:23.277838 2434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:23.285372 2434 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:37153
I20260928 14:40:23.285925 2434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:40:23.287873 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 2434
I20260928 14:40:23.287984 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:40:23.288177 22581 external_mini_cluster.cc:1428] Setting key 5302fe6ddd9166bf28e8fa25d7bb080a
Sep 28 14:40:23 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606423, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:23.302352 2562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:37153 every 8 connection(s)
I20260928 14:40:23.303233 2565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:23.286920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:35737 (local address 127.22.13.65:34489)
0928 14:40:23.287158 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:23.287216 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:23.288730 (+ 1514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:23.291150 (+ 2420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:23.291156 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:23.291443 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:23.291764 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:23.291772 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:23.292834 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:23.292835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:23.294206 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:23.294211 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:23.294782 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:23.294784 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:23.294945 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:23.295661 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:23.295673 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:23.298333 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:23.301000 (+ 2667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:23.301015 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:23.301018 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:23.301425 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:23.301738 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:23.301738 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:23.301739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:23.301960 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:23.302383 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:23.302393 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:23.302629 (+ 236us) client_negotiation.cc:769] Sending connection context
0928 14:40:23.302760 (+ 131us) client_negotiation.cc:240] Negotiation successful
0928 14:40:23.302976 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":108,"threads_started":1}
I20260928 14:40:23.303690 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:23.287212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:34489 (local address 127.22.13.126:35737)
0928 14:40:23.290824 (+ 3612us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:23.290827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:23.290842 (+ 15us) server_negotiation.cc:407] Connection header received
0928 14:40:23.290875 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:23.290876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:23.290895 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:23.290999 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:23.291901 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:23.292700 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:23.294325 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:23.294631 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:23.294702 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:23.298521 (+ 3819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:23.298523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:23.298523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:23.298534 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:23.300837 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:23.301547 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:23.301548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:23.301549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:23.301632 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:23.302067 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:23.302068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:23.302069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:23.302262 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:23.302346 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:23.303440 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0928 14:40:23.303568 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1739,"thread_start_us":82,"threads_started":1}
I20260928 14:40:23.304250 2563 heartbeater.cc:347] Connected to a master server at 127.22.13.126:35737
I20260928 14:40:23.304363 2563 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:23.304616 2563 heartbeater.cc:511] Master 127.22.13.126:35737 requested a full tablet report, sending...
I20260928 14:40:23.305249 2353 ts_manager.cc:194] Registered new tserver with Master: eaf2b4a72dd3425ea0889445dda7b897 (127.22.13.65:37153)
I20260928 14:40:23.306211 2353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:34489
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:40:23.335068 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.13.84:45063/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35737
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:23.456985 2571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:23.457414 2571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:23.457504 2571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:23.459569 2571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:23.459753 2571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:23.459841 2571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:40:23.462493 2571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.13.84:45063/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:23.463136 2571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:23.463526 2571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:23.470762 2578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:23.479722 2571 server_base.cc:1034] running on GCE node
W20260928 14:40:23.480038 2581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:23.480667 2579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:23.481031 2571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:23.481360 2571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:23.482510 2571 hybrid_clock.cc:648] HybridClock initialized: now 1790606423482498 us; error 54 us; skew 500 ppm
Sep 28 14:40:23 dist-test-slave-wwwb krb5kdc[1877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606423, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:23.486188 2571 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:40:23.486780 2571 webserver.cc:460] Webserver started at http://127.22.13.66:37011/ using document root <none> and password file <none>
I20260928 14:40:23.487061 2571 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:23.487112 2571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:23.487242 2571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:40:23 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606423, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:40:23.510387 2571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "938ece79da564d42858dbda7e210b950"
format_stamp: "Formatted at 2026-09-28 14:40:23 on dist-test-slave-wwwb"
server_key: "96a2f1e2ff2792c666ba693d024ba137"
server_key_iv: "4bf619cecda42d58e089b891130c9d55"
server_key_version: "kuduclusterkey@0"
I20260928 14:40:23.510994 2571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "938ece79da564d42858dbda7e210b950"
format_stamp: "Formatted at 2026-09-28 14:40:23 on dist-test-slave-wwwb"
server_key: "96a2f1e2ff2792c666ba693d024ba137"
server_key_iv: "4bf619cecda42d58e089b891130c9d55"
server_key_version: "kuduclusterkey@0"
I20260928 14:40:23.518524 2571 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.000s
I20260928 14:40:23.541533 2588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:23.544037 2571 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260928 14:40:23.544358 2571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "938ece79da564d42858dbda7e210b950"
format_stamp: "Formatted at 2026-09-28 14:40:23 on dist-test-slave-wwwb"
server_key: "96a2f1e2ff2792c666ba693d024ba137"
server_key_iv: "4bf619cecda42d58e089b891130c9d55"
server_key_version: "kuduclusterkey@0"
I20260928 14:40:23.544553 2571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 28 14:40:23 dist-test-slave-wwwb krb5kdc[1877](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790606423, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:23.577072 2571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:23.586629 2571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:23.586921 2571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:23.587239 2571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:23.587674 2571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:40:23.587779 2571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:23.587852 2571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:40:23.587913 2571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:23.601419 2571 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:33419
I20260928 14:40:23.602032 2571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:40:23.603789 2701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:33419 every 8 connection(s)
I20260928 14:40:23.609784 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 2571
I20260928 14:40:23.609882 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
Sep 28 14:40:23 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606423, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:23.610163 22581 external_mini_cluster.cc:1428] Setting key bc88dbc8d50db8ec4c90431728618b1d
I20260928 14:40:23.615314 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:23.603633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:46143 (local address 127.22.13.126:35737)
0928 14:40:23.603706 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:23.603708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:23.603976 (+ 268us) server_negotiation.cc:407] Connection header received
0928 14:40:23.604716 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:23.604718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:23.604736 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:23.604770 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:23.605669 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:23.606352 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:23.607589 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:23.607899 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:23.608015 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:23.611368 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:23.611369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:23.611370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:23.611378 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:23.613342 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:23.614013 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:23.614014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:23.614015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:23.614077 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:23.614410 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:23.614411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:23.614411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:23.614581 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:23.614626 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:23.615038 (+ 412us) server_negotiation.cc:299] Negotiation successful
0928 14:40:23.615147 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:40:23.615934 2704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:23.603329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:35737 (local address 127.22.13.66:46143)
0928 14:40:23.603765 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:23.603818 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:23.604558 (+ 740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:23.604973 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:23.604977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:23.605244 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:23.605545 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:23.605553 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:23.606451 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:23.606452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:23.607483 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:23.607486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:23.608039 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:23.608040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:23.608192 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:23.608797 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:23.608807 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:23.611247 (+ 2440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:23.613506 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:23.613517 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:23.613521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:23.613890 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:23.614149 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:23.614150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:23.614150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:23.614333 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:23.614673 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:23.614681 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:23.614879 (+ 198us) client_negotiation.cc:769] Sending connection context
0928 14:40:23.615438 (+ 559us) client_negotiation.cc:240] Negotiation successful
0928 14:40:23.615643 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":92,"threads_started":1}
I20260928 14:40:23.616660 2702 heartbeater.cc:347] Connected to a master server at 127.22.13.126:35737
I20260928 14:40:23.616802 2702 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:23.617003 2702 heartbeater.cc:511] Master 127.22.13.126:35737 requested a full tablet report, sending...
I20260928 14:40:23.617450 2353 ts_manager.cc:194] Registered new tserver with Master: 938ece79da564d42858dbda7e210b950 (127.22.13.66:33419)
I20260928 14:40:23.617864 2353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:46143
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:40:23.644276 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.13.84:45063/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35737
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:23.778085 2708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:23.778486 2708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:23.778612 2708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:23.780696 2708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:23.780851 2708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:23.780968 2708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:40:23.783710 2708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.13.84:45063/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:23.784410 2708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:23.784813 2708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:23.791945 2714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:23.792440 2713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:23.792713 2716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:23.793974 2708 server_base.cc:1034] running on GCE node
I20260928 14:40:23.794288 2708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:23.794698 2708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:23.795964 2708 hybrid_clock.cc:648] HybridClock initialized: now 1790606423795937 us; error 48 us; skew 500 ppm
Sep 28 14:40:23 dist-test-slave-wwwb krb5kdc[1877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606423, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:23.799993 2708 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:40:23.800653 2708 webserver.cc:460] Webserver started at http://127.22.13.67:41483/ using document root <none> and password file <none>
I20260928 14:40:23.800954 2708 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:23.801019 2708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:23.801165 2708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:40:23 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606423, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:40:23.819284 2708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "2de369bf2ce34983a9c0ff22a30cec79"
format_stamp: "Formatted at 2026-09-28 14:40:23 on dist-test-slave-wwwb"
server_key: "5dd26635483189abf66d076b63ea200a"
server_key_iv: "07b5d8e5aa95c06287ad0d057648f609"
server_key_version: "kuduclusterkey@0"
I20260928 14:40:23.819918 2708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "2de369bf2ce34983a9c0ff22a30cec79"
format_stamp: "Formatted at 2026-09-28 14:40:23 on dist-test-slave-wwwb"
server_key: "5dd26635483189abf66d076b63ea200a"
server_key_iv: "07b5d8e5aa95c06287ad0d057648f609"
server_key_version: "kuduclusterkey@0"
I20260928 14:40:23.822059 2708 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260928 14:40:23.839855 2723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:23.840154 2708 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260928 14:40:23.840263 2708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "2de369bf2ce34983a9c0ff22a30cec79"
format_stamp: "Formatted at 2026-09-28 14:40:23 on dist-test-slave-wwwb"
server_key: "5dd26635483189abf66d076b63ea200a"
server_key_iv: "07b5d8e5aa95c06287ad0d057648f609"
server_key_version: "kuduclusterkey@0"
I20260928 14:40:23.840368 2708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:23.867789 2708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:23.870715 2708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:23.870971 2708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:23.871282 2708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:23.871755 2708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:40:23.871856 2708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:23.871937 2708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:40:23.871999 2708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:23.879293 2708 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:33911
I20260928 14:40:23.879952 2708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:40:23.884414 2836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:33911 every 8 connection(s)
I20260928 14:40:23.890092 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 2708
I20260928 14:40:23.890213 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:40:23.890404 22581 external_mini_cluster.cc:1428] Setting key 77f84c1f621ba381dc472d4149c00a20
Sep 28 14:40:23 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606423, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:23.897336 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:23.885129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:47141 (local address 127.22.13.126:35737)
0928 14:40:23.885186 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:23.885189 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:23.885697 (+ 508us) server_negotiation.cc:407] Connection header received
0928 14:40:23.886472 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:23.886474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:23.886489 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:23.886521 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:23.887450 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:23.888207 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:23.889773 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:23.890115 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:23.890218 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:23.893099 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:23.893100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:23.893101 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:23.893110 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:23.895399 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:23.896068 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:23.896070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:23.896070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:23.896149 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:23.896499 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:23.896500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:23.896500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:23.896660 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:23.896734 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:23.897126 (+ 392us) server_negotiation.cc:299] Negotiation successful
0928 14:40:23.897234 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 14:40:23.897553 2839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:23.885129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:35737 (local address 127.22.13.67:47141)
0928 14:40:23.885558 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:23.885603 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:23.886330 (+ 727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:23.886745 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:23.886749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:23.887019 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:23.887330 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:23.887337 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:23.888336 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:23.888337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:23.889632 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:23.889636 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:23.890324 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:23.890326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:23.890497 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:23.891133 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:23.891145 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:23.892970 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:23.895533 (+ 2563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:23.895548 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:23.895552 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:23.895960 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:23.896232 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:23.896233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:23.896234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:23.896419 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:23.896737 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:23.896745 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:23.896957 (+ 212us) client_negotiation.cc:769] Sending connection context
0928 14:40:23.897090 (+ 133us) client_negotiation.cc:240] Negotiation successful
0928 14:40:23.897291 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":98,"threads_started":1}
I20260928 14:40:23.898221 2837 heartbeater.cc:347] Connected to a master server at 127.22.13.126:35737
I20260928 14:40:23.898353 2837 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:23.898592 2837 heartbeater.cc:511] Master 127.22.13.126:35737 requested a full tablet report, sending...
I20260928 14:40:23.899055 2353 ts_manager.cc:194] Registered new tserver with Master: 2de369bf2ce34983a9c0ff22a30cec79 (127.22.13.67:33911)
I20260928 14:40:23.899541 2353 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:47141
I20260928 14:40:23.902710 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:40:23.903429 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 2434
I20260928 14:40:23.910631 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 2571
I20260928 14:40:23.915853 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 2708
I20260928 14:40:23.931649 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 2321
I20260928 14:40:23.939177 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:35737
--webserver_interface=127.22.13.126
--webserver_port=45193
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:35737
--ranger_config_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.13.84:45063/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.13.84:45063/kms with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:24.051785 2840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:24.052026 2840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:24.052052 2840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:24.053354 2840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:40:24.053406 2840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:24.053421 2840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:24.053433 2840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:40:24.053447 2840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:40:24.055397 2840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.13.84:45063/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:35737
--ranger_config_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:35737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=45193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:24.055740 2840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:24.055995 2840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:24.061116 2846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:24.061131 2845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:24.061136 2848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:24.061390 2840 server_base.cc:1034] running on GCE node
I20260928 14:40:24.061648 2840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:24.062089 2840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:24.063262 2840 hybrid_clock.cc:648] HybridClock initialized: now 1790606424063242 us; error 53 us; skew 500 ppm
Sep 28 14:40:24 dist-test-slave-wwwb krb5kdc[1877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606424, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:24.066154 2840 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:40:24.066654 2840 webserver.cc:460] Webserver started at http://127.22.13.126:45193/ using document root <none> and password file <none>
I20260928 14:40:24.066918 2840 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:24.066998 2840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 14:40:24 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606424, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:40:24.128252 2840 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:24.129056 2856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:24.129263 2840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:24.129380 2840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "909a182c9df043a5b0c506cbbc4d6c63"
format_stamp: "Formatted at 2026-09-28 14:40:22 on dist-test-slave-wwwb"
server_key: "1652fab200a02b6d3465d4b60ac5d4bc"
server_key_iv: "fa739b89072e6d6135e3c6bec6ac883b"
server_key_version: "kuduclusterkey@0"
I20260928 14:40:24.129793 2840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:24.140604 2840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:24.142719 2840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:24.142879 2840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:24.147981 2840 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:35737
I20260928 14:40:24.148023 2908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:35737 every 8 connection(s)
I20260928 14:40:24.148468 2840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:40:24.149818 2909 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:40:24.150555 2909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63: Bootstrap starting.
I20260928 14:40:24.157292 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 2840
I20260928 14:40:24.157624 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.13.84:45063/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:37153
--local_ip_for_outbound_sockets=127.22.13.65
--tserver_master_addrs=127.22.13.126:35737
--webserver_port=39295
--webserver_interface=127.22.13.65
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.13.84:45063/kms with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:40:24.159173 2909 log.cc:824] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:24.161015 2909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:40:24.161360 2909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63: Bootstrap complete.
I20260928 14:40:24.164032 2909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "909a182c9df043a5b0c506cbbc4d6c63" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35737 } }
I20260928 14:40:24.164328 2909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 909a182c9df043a5b0c506cbbc4d6c63, State: Initialized, Role: FOLLOWER
I20260928 14:40:24.164543 2909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "909a182c9df043a5b0c506cbbc4d6c63" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35737 } }
I20260928 14:40:24.164706 2909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:40:24.164777 2909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:40:24.164834 2909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:40:24.166042 2909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "909a182c9df043a5b0c506cbbc4d6c63" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35737 } }
I20260928 14:40:24.166167 2909 leader_election.cc:302] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 909a182c9df043a5b0c506cbbc4d6c63; no voters:
I20260928 14:40:24.166386 2909 leader_election.cc:288] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 14:40:24.166425 2913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 14:40:24.166635 2913 raft_consensus.cc:686] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [term 2 LEADER]: Becoming Leader. State: Replica: 909a182c9df043a5b0c506cbbc4d6c63, State: Running, Role: LEADER
I20260928 14:40:24.166776 2913 consensus_queue.cc:237] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "909a182c9df043a5b0c506cbbc4d6c63" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35737 } }
I20260928 14:40:24.166829 2909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:40:24.167076 2913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "909a182c9df043a5b0c506cbbc4d6c63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "909a182c9df043a5b0c506cbbc4d6c63" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35737 } } }
I20260928 14:40:24.167183 2913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:24.167315 2913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 909a182c9df043a5b0c506cbbc4d6c63. Latest consensus state: current_term: 2 leader_uuid: "909a182c9df043a5b0c506cbbc4d6c63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "909a182c9df043a5b0c506cbbc4d6c63" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35737 } } }
I20260928 14:40:24.167392 2913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63 [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:24.168597 2909 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 14:40:24.171802 2917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:40:24.172648 2917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:40:24.172883 2917 catalog_manager.cc:1259] Loaded cluster ID: 6351865a421b40aeb43d831c42ad2e80
I20260928 14:40:24.172923 2917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:40:24.174998 2917 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:40:24.175262 2917 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63: Loaded TSK: 0
I20260928 14:40:24.175516 2917 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 14:40:24.277163 2911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:24.277436 2911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:24.277478 2911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:24.278847 2911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:24.278925 2911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:24.278976 2911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:40:24.280625 2911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.13.84:45063/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:37153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=39295
--tserver_master_addrs=127.22.13.126:35737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:24.280921 2911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:24.281183 2911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:24.288080 2949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:24.289757 2947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:24.290089 2946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:24.290179 2911 server_base.cc:1034] running on GCE node
I20260928 14:40:24.290551 2911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:24.290880 2911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:24.292060 2911 hybrid_clock.cc:648] HybridClock initialized: now 1790606424292035 us; error 41 us; skew 500 ppm
Sep 28 14:40:24 dist-test-slave-wwwb krb5kdc[1877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606424, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:24.295745 2911 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:40:24.296365 2911 webserver.cc:460] Webserver started at http://127.22.13.65:39295/ using document root <none> and password file <none>
I20260928 14:40:24.296680 2911 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:24.296826 2911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 14:40:24 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606424, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:40:24.314426 2911 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260928 14:40:24.322461 2957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:24.322731 2911 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.000s
I20260928 14:40:24.322849 2911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "eaf2b4a72dd3425ea0889445dda7b897"
format_stamp: "Formatted at 2026-09-28 14:40:23 on dist-test-slave-wwwb"
server_key: "7928d447f7bb4c9502c2d00ffd912220"
server_key_iv: "25e062b8ffa0007cc7407158a02a9444"
server_key_version: "kuduclusterkey@0"
I20260928 14:40:24.323372 2911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:24.339792 2911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:24.342412 2911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:24.342644 2911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:24.342959 2911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:24.343420 2911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:40:24.343525 2911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:24.343639 2911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:40:24.343730 2911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:24.352487 2911 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:37153
I20260928 14:40:24.352574 3070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:37153 every 8 connection(s)
I20260928 14:40:24.353310 2911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:40:24.359002 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 2911
I20260928 14:40:24.359314 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.13.84:45063/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:33419
--local_ip_for_outbound_sockets=127.22.13.66
--tserver_master_addrs=127.22.13.126:35737
--webserver_port=37011
--webserver_interface=127.22.13.66
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.13.84:45063/kms with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:40:24 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606424, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:24.372103 3074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:24.354465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:35619 (local address 127.22.13.126:35737)
0928 14:40:24.359396 (+ 4931us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:24.359407 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:24.359452 (+ 45us) server_negotiation.cc:407] Connection header received
0928 14:40:24.359621 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:24.359628 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:24.359690 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:24.359844 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:24.360752 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:24.362238 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:24.363743 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:24.364453 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:24.364575 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:24.367658 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:24.367674 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:24.367674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:24.367689 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:24.369759 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:24.370351 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:24.370355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:24.370356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:24.370471 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:24.370760 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:24.370760 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:24.370761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:24.371080 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:24.371153 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:24.371593 (+ 440us) server_negotiation.cc:299] Negotiation successful
0928 14:40:24.371789 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4857,"thread_start_us":89,"threads_started":1}
I20260928 14:40:24.373919 3073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:24.354368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:35737 (local address 127.22.13.65:35619)
0928 14:40:24.358359 (+ 3991us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:24.358417 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:24.359125 (+ 708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:24.360021 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:24.360026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:24.360292 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:24.360595 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:24.360602 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:24.362373 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:24.362374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:24.363635 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:24.363638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:24.364600 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:24.364601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:24.364764 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:24.365402 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:24.365415 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:24.367486 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:24.369875 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:24.369884 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:24.369886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:24.370246 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:24.370545 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:24.370546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:24.370546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:24.370684 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:24.371193 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:24.371200 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:24.371368 (+ 168us) client_negotiation.cc:769] Sending connection context
0928 14:40:24.372736 (+ 1368us) client_negotiation.cc:240] Negotiation successful
0928 14:40:24.373466 (+ 730us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3938,"thread_start_us":92,"threads_started":1}
I20260928 14:40:24.375165 3071 heartbeater.cc:347] Connected to a master server at 127.22.13.126:35737
I20260928 14:40:24.375360 3071 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:24.375686 3071 heartbeater.cc:511] Master 127.22.13.126:35737 requested a full tablet report, sending...
W20260928 14:40:24.376677 3071 heartbeater.cc:644] Failed to heartbeat to 127.22.13.126:35737 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 14:40:24.376853 3071 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:24.376981 3071 heartbeater.cc:511] Master 127.22.13.126:35737 requested a full tablet report, sending...
I20260928 14:40:24.377370 3071 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:24.377506 3071 heartbeater.cc:511] Master 127.22.13.126:35737 requested a full tablet report, sending...
W20260928 14:40:24.377825 3071 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:40:24.490957 3075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:24.491295 3075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:24.491345 3075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:24.493382 3075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:24.493460 3075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:24.493510 3075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:40:24.496145 3075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.13.84:45063/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:33419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=37011
--tserver_master_addrs=127.22.13.126:35737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:24.496623 3075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:24.496965 3075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:24.503991 3083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:24.503991 3081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:24.504485 3080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:24.504833 3075 server_base.cc:1034] running on GCE node
I20260928 14:40:24.505088 3075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:24.505425 3075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:24.506564 3075 hybrid_clock.cc:648] HybridClock initialized: now 1790606424506544 us; error 36 us; skew 500 ppm
Sep 28 14:40:24 dist-test-slave-wwwb krb5kdc[1877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606424, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:24.510075 3075 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:40:24.510720 3075 webserver.cc:460] Webserver started at http://127.22.13.66:37011/ using document root <none> and password file <none>
I20260928 14:40:24.511024 3075 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:24.511163 3075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 14:40:24 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606424, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:40:24.536106 3075 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:24.537065 3091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:24.537238 3075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:24.537359 3075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "938ece79da564d42858dbda7e210b950"
format_stamp: "Formatted at 2026-09-28 14:40:23 on dist-test-slave-wwwb"
server_key: "96a2f1e2ff2792c666ba693d024ba137"
server_key_iv: "4bf619cecda42d58e089b891130c9d55"
server_key_version: "kuduclusterkey@0"
I20260928 14:40:24.537772 3075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:24.549674 3075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:24.552246 3075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:24.552462 3075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:24.552775 3075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:24.553212 3075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:40:24.553270 3075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:24.553345 3075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:40:24.553378 3075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:24.561652 3075 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:33419
I20260928 14:40:24.561769 3204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:33419 every 8 connection(s)
I20260928 14:40:24.562325 3075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
Sep 28 14:40:24 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606424, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:24.570508 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 3075
I20260928 14:40:24.570811 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.13.84:45063/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:33911
--local_ip_for_outbound_sockets=127.22.13.67
--tserver_master_addrs=127.22.13.126:35737
--webserver_port=41483
--webserver_interface=127.22.13.67
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.13.84:45063/kms with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:40:24.577196 3074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:24.563282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:58323 (local address 127.22.13.126:35737)
0928 14:40:24.563343 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:24.563345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:24.563690 (+ 345us) server_negotiation.cc:407] Connection header received
0928 14:40:24.564433 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:24.564435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:24.564454 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:24.564550 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:24.565354 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:24.566214 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:24.567812 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:24.568158 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:24.568265 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:24.572791 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:24.572794 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:24.572794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:24.572807 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:24.574971 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:24.575712 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:24.575714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:24.575714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:24.575792 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:24.576170 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:24.576171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:24.576172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:24.576351 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:24.576407 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:24.576917 (+ 510us) server_negotiation.cc:299] Negotiation successful
0928 14:40:24.577045 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 14:40:24.578013 3207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:24.563296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:35737 (local address 127.22.13.66:58323)
0928 14:40:24.563511 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:24.563563 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:24.564279 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:24.564632 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:24.564637 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:24.564895 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:24.565210 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:24.565218 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:24.566353 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:24.566354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:24.567705 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:24.567710 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:24.568302 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:24.568303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:24.568459 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:24.569131 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:24.569141 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:24.572597 (+ 3456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:24.575119 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:24.575133 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:24.575137 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:24.575584 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:24.575887 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:24.575887 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:24.575888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:24.576074 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:24.576470 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:24.576492 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:24.576717 (+ 225us) client_negotiation.cc:769] Sending connection context
0928 14:40:24.577395 (+ 678us) client_negotiation.cc:240] Negotiation successful
0928 14:40:24.577645 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":97,"threads_started":1}
I20260928 14:40:24.578629 3205 heartbeater.cc:347] Connected to a master server at 127.22.13.126:35737
I20260928 14:40:24.578753 3205 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:24.578994 3205 heartbeater.cc:511] Master 127.22.13.126:35737 requested a full tablet report, sending...
W20260928 14:40:24.579823 3205 heartbeater.cc:644] Failed to heartbeat to 127.22.13.126:35737 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 14:40:24.579910 3205 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:24.579993 3205 heartbeater.cc:511] Master 127.22.13.126:35737 requested a full tablet report, sending...
I20260928 14:40:24.580428 3205 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:24.580531 3205 heartbeater.cc:511] Master 127.22.13.126:35737 requested a full tablet report, sending...
W20260928 14:40:24.580945 3205 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:40:24.682219 3208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:24.682487 3208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:24.682533 3208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:24.683924 3208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:24.684017 3208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:24.684072 3208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:40:24.685801 3208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.13.84:45063/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:33911
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=41483
--tserver_master_addrs=127.22.13.126:35737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:24.686161 3208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:24.686453 3208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:24.692022 3216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:24.692894 3213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:24.693317 3214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:24.693636 3208 server_base.cc:1034] running on GCE node
I20260928 14:40:24.694031 3208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:24.694371 3208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:24.695580 3208 hybrid_clock.cc:648] HybridClock initialized: now 1790606424695536 us; error 63 us; skew 500 ppm
Sep 28 14:40:24 dist-test-slave-wwwb krb5kdc[1877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606424, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:24.698791 3208 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:40:24.699312 3208 webserver.cc:460] Webserver started at http://127.22.13.67:41483/ using document root <none> and password file <none>
I20260928 14:40:24.699641 3208 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:24.699728 3208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 14:40:24 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606424, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:40:24.725364 3208 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260928 14:40:24.726646 3225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:24.726852 3208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:40:24.726963 3208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "2de369bf2ce34983a9c0ff22a30cec79"
format_stamp: "Formatted at 2026-09-28 14:40:23 on dist-test-slave-wwwb"
server_key: "5dd26635483189abf66d076b63ea200a"
server_key_iv: "07b5d8e5aa95c06287ad0d057648f609"
server_key_version: "kuduclusterkey@0"
I20260928 14:40:24.727483 3208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:24.737208 3208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:24.739876 3208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:24.740073 3208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:24.740429 3208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:24.740967 3208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:40:24.741067 3208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:24.741124 3208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:40:24.741159 3208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:24.748750 3208 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:33911
I20260928 14:40:24.748852 3338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:33911 every 8 connection(s)
I20260928 14:40:24.749277 3208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:40:24.750547 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 3208
Sep 28 14:40:24 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606424, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:24.761214 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:24.750199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:35737 (local address 127.22.13.67:59715)
0928 14:40:24.750388 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:24.750448 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:24.751184 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:24.751503 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:24.751507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:24.751716 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:24.751991 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:24.751999 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:24.752954 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:24.752955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:24.754233 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:24.754237 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:24.754793 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:24.754794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:24.754954 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:24.755638 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:24.755650 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:24.757344 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:24.759374 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:24.759385 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:24.759388 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:24.759734 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:24.759978 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:24.759979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:24.759979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:24.760159 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:24.760486 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:24.760495 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:24.760692 (+ 197us) client_negotiation.cc:769] Sending connection context
0928 14:40:24.760813 (+ 121us) client_negotiation.cc:240] Negotiation successful
0928 14:40:24.760982 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260928 14:40:24.761561 3074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:24.750198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:59715 (local address 127.22.13.126:35737)
0928 14:40:24.750247 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:24.750249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:24.750572 (+ 323us) server_negotiation.cc:407] Connection header received
0928 14:40:24.751301 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:24.751302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:24.751316 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:24.751385 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:24.752109 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:24.752823 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:24.754326 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:24.754613 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:24.754689 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:24.757477 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:24.757479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:24.757479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:24.757489 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:24.759267 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:24.759835 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:24.759836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:24.759836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:24.759902 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:24.760243 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:24.760244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:24.760244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:24.760389 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:24.760456 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:24.761381 (+ 925us) server_negotiation.cc:299] Negotiation successful
0928 14:40:24.761473 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:40:24.762100 3339 heartbeater.cc:347] Connected to a master server at 127.22.13.126:35737
I20260928 14:40:24.762199 3339 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:24.762405 3339 heartbeater.cc:511] Master 127.22.13.126:35737 requested a full tablet report, sending...
W20260928 14:40:24.763083 3339 heartbeater.cc:644] Failed to heartbeat to 127.22.13.126:35737 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 14:40:24.763166 3339 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:24.763245 3339 heartbeater.cc:511] Master 127.22.13.126:35737 requested a full tablet report, sending...
I20260928 14:40:24.763578 3339 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:24.763660 3339 heartbeater.cc:511] Master 127.22.13.126:35737 requested a full tablet report, sending...
W20260928 14:40:24.763967 3339 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 14:40:24.769665 3343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:24.750959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37860 (local address 127.22.13.126:35737)
0928 14:40:24.751086 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:24.751087 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:24.754007 (+ 2920us) server_negotiation.cc:407] Connection header received
0928 14:40:24.754041 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:24.754042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:24.754059 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:24.754142 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:24.758405 (+ 4263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:24.760760 (+ 2355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:24.761911 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:24.762292 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:24.762366 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:24.766431 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:24.766433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:24.766434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:24.766442 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:24.768332 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:24.768789 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:24.768790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:24.768791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:24.768860 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:24.769075 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:24.769076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:24.769076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:24.769212 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:24.769410 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:24.769451 (+ 41us) server_negotiation.cc:299] Negotiation successful
0928 14:40:24.769544 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
Sep 28 14:40:25 dist-test-slave-wwwb krb5kdc[1877](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790606425, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:25.378067 3071 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:25.378257 3071 heartbeater.cc:511] Master 127.22.13.126:35737 requested a full tablet report, sending...
I20260928 14:40:25.378764 2873 ts_manager.cc:194] Registered new tserver with Master: eaf2b4a72dd3425ea0889445dda7b897 (127.22.13.65:37153)
I20260928 14:40:25.379746 2873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:35619
I20260928 14:40:25.581157 3205 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:25.581318 3205 heartbeater.cc:511] Master 127.22.13.126:35737 requested a full tablet report, sending...
I20260928 14:40:25.581756 2873 ts_manager.cc:194] Registered new tserver with Master: 938ece79da564d42858dbda7e210b950 (127.22.13.66:33419)
I20260928 14:40:25.582322 2873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:58323
Sep 28 14:40:25 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606425, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:40:25.764153 3339 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:25.764307 3339 heartbeater.cc:511] Master 127.22.13.126:35737 requested a full tablet report, sending...
I20260928 14:40:25.764590 2872 ts_manager.cc:194] Registered new tserver with Master: 2de369bf2ce34983a9c0ff22a30cec79 (127.22.13.67:33911)
I20260928 14:40:25.765081 2872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:59715
I20260928 14:40:25.774024 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:40:25.788259 3360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:25.775324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37872 (local address 127.22.13.126:35737)
0928 14:40:25.775600 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:25.775602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:25.776058 (+ 456us) server_negotiation.cc:407] Connection header received
0928 14:40:25.776599 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:25.776601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:25.776617 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:25.777181 (+ 564us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:25.780898 (+ 3717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:25.781684 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:25.782715 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:25.783036 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:25.783110 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:25.784898 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:25.784900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:25.784900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:25.784910 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:25.786866 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:25.787312 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:25.787314 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:25.787314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:25.787382 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:25.787609 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:25.787610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:25.787610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:25.787785 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:25.787901 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:25.788044 (+ 143us) server_negotiation.cc:299] Negotiation successful
0928 14:40:25.788142 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":60,"threads_started":1}
I20260928 14:40:25.790755 2872 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:40:25.793223 2872 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:40:25.823058 3369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:25.811978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:33911 (local address 127.0.0.1:54004)
0928 14:40:25.818086 (+ 6108us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:25.818154 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:25.818253 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:25.818818 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:25.818822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:25.818825 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:25.818910 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:25.818912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:25.820232 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:25.820233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:25.821562 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:25.821565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:25.822716 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:25.822717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:25.822803 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:25.822820 (+ 17us) client_negotiation.cc:769] Sending connection context
0928 14:40:25.822896 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 14:40:25.822953 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6056,"thread_start_us":104,"threads_started":1}
I20260928 14:40:25.823443 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:25.812471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54004 (local address 127.22.13.67:33911)
0928 14:40:25.815182 (+ 2711us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:25.815200 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:25.818496 (+ 3296us) server_negotiation.cc:407] Connection header received
0928 14:40:25.818544 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:25.818547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:25.818565 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:25.818645 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:25.818992 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:25.820105 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:25.821663 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:25.822522 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:25.822606 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:25.822677 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:25.823312 (+ 635us) server_negotiation.cc:299] Negotiation successful
0928 14:40:25.823368 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2664,"thread_start_us":52,"threads_started":1}
I20260928 14:40:25.824347 3273 tablet_service.cc:1467] Processing CreateTablet for tablet 628fce7b1fa3416180559820c01de0c5 (DEFAULT_TABLE table=test-table [id=212bc0d75f9740d2a827689206059b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:40:25.824764 3273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 628fce7b1fa3416180559820c01de0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:25.828552 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:25.812028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:37153 (local address 127.0.0.1:56500)
0928 14:40:25.820259 (+ 8231us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:25.820270 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:25.820316 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:25.822156 (+ 1840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:25.822157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:25.822159 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:25.822209 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:25.822210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:25.824069 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:25.824070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:25.825126 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:25.825129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:25.828308 (+ 3179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:25.828309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:25.828371 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:25.828375 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:40:25.828428 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 14:40:25.828472 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8207,"thread_start_us":99,"threads_started":1}
I20260928 14:40:25.829419 3368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:25.812128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56500 (local address 127.22.13.65:37153)
0928 14:40:25.818413 (+ 6285us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:25.818420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:25.820381 (+ 1961us) server_negotiation.cc:407] Connection header received
0928 14:40:25.820424 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:25.820428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:25.820445 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:25.820511 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:25.822289 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:25.823978 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:25.826120 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:25.826873 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:25.826948 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:25.827006 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:25.829288 (+ 2282us) server_negotiation.cc:299] Negotiation successful
0928 14:40:25.829343 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6243,"thread_start_us":58,"threads_started":1}
I20260928 14:40:25.829667 3370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:25.812343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50224 (local address 127.22.13.66:33419)
0928 14:40:25.818768 (+ 6425us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:25.818773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:25.820724 (+ 1951us) server_negotiation.cc:407] Connection header received
0928 14:40:25.820762 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:25.820764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:25.820780 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:25.820840 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:25.821028 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:25.822059 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:25.827311 (+ 5252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:25.828125 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:25.828214 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:25.828270 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:25.829550 (+ 1280us) server_negotiation.cc:299] Negotiation successful
0928 14:40:25.829600 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5580,"thread_start_us":63,"threads_started":1}
I20260928 14:40:25.829218 3366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:25.812254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:33419 (local address 127.0.0.1:50224)
0928 14:40:25.820617 (+ 8363us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:25.820626 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:25.820664 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:25.820921 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:25.820922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:25.820924 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:25.820966 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:25.820967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:25.823366 (+ 2399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:25.823367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:25.827209 (+ 3842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:25.827213 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:25.828985 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:25.828986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:25.829050 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:25.829054 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:40:25.829100 (+ 46us) client_negotiation.cc:240] Negotiation successful
0928 14:40:25.829140 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8347,"thread_start_us":58,"threads_started":1}
I20260928 14:40:25.830567 3005 tablet_service.cc:1467] Processing CreateTablet for tablet 628fce7b1fa3416180559820c01de0c5 (DEFAULT_TABLE table=test-table [id=212bc0d75f9740d2a827689206059b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:40:25.830694 3139 tablet_service.cc:1467] Processing CreateTablet for tablet 628fce7b1fa3416180559820c01de0c5 (DEFAULT_TABLE table=test-table [id=212bc0d75f9740d2a827689206059b02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:40:25.830945 3005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 628fce7b1fa3416180559820c01de0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:25.831046 3139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 628fce7b1fa3416180559820c01de0c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:25.834985 3373 tablet_bootstrap.cc:492] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950: Bootstrap starting.
I20260928 14:40:25.835651 3373 tablet_bootstrap.cc:654] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:25.835963 3373 log.cc:824] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:25.836355 3372 tablet_bootstrap.cc:492] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79: Bootstrap starting.
I20260928 14:40:25.836830 3373 tablet_bootstrap.cc:492] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950: No bootstrap required, opened a new log
I20260928 14:40:25.836901 3373 ts_tablet_manager.cc:1397] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:40:25.838719 3372 tablet_bootstrap.cc:654] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:25.839148 3372 log.cc:824] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:25.839586 3373 raft_consensus.cc:348] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de369bf2ce34983a9c0ff22a30cec79" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33911 } } peers { permanent_uuid: "938ece79da564d42858dbda7e210b950" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33419 } } peers { permanent_uuid: "eaf2b4a72dd3425ea0889445dda7b897" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37153 } }
I20260928 14:40:25.839768 3373 raft_consensus.cc:374] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:25.839802 3373 raft_consensus.cc:729] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 938ece79da564d42858dbda7e210b950, State: Initialized, Role: FOLLOWER
I20260928 14:40:25.839984 3373 consensus_queue.cc:260] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de369bf2ce34983a9c0ff22a30cec79" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33911 } } peers { permanent_uuid: "938ece79da564d42858dbda7e210b950" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33419 } } peers { permanent_uuid: "eaf2b4a72dd3425ea0889445dda7b897" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37153 } }
I20260928 14:40:25.840337 3373 ts_tablet_manager.cc:1428] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:40:25.840381 3205 heartbeater.cc:503] Master 127.22.13.126:35737 was elected leader, sending a full tablet report...
I20260928 14:40:25.840586 3374 tablet_bootstrap.cc:492] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897: Bootstrap starting.
I20260928 14:40:25.841504 3374 tablet_bootstrap.cc:654] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:25.841861 3374 log.cc:824] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:25.842804 3374 tablet_bootstrap.cc:492] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897: No bootstrap required, opened a new log
I20260928 14:40:25.842901 3374 ts_tablet_manager.cc:1397] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260928 14:40:25.845407 3374 raft_consensus.cc:348] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de369bf2ce34983a9c0ff22a30cec79" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33911 } } peers { permanent_uuid: "938ece79da564d42858dbda7e210b950" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33419 } } peers { permanent_uuid: "eaf2b4a72dd3425ea0889445dda7b897" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37153 } }
I20260928 14:40:25.845582 3374 raft_consensus.cc:374] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:25.845613 3374 raft_consensus.cc:729] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf2b4a72dd3425ea0889445dda7b897, State: Initialized, Role: FOLLOWER
I20260928 14:40:25.845818 3374 consensus_queue.cc:260] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de369bf2ce34983a9c0ff22a30cec79" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33911 } } peers { permanent_uuid: "938ece79da564d42858dbda7e210b950" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33419 } } peers { permanent_uuid: "eaf2b4a72dd3425ea0889445dda7b897" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37153 } }
I20260928 14:40:25.846119 3374 ts_tablet_manager.cc:1428] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 14:40:25.846185 3071 heartbeater.cc:503] Master 127.22.13.126:35737 was elected leader, sending a full tablet report...
I20260928 14:40:25.846643 3372 tablet_bootstrap.cc:492] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79: No bootstrap required, opened a new log
I20260928 14:40:25.846716 3372 ts_tablet_manager.cc:1397] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260928 14:40:25.848754 3372 raft_consensus.cc:348] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de369bf2ce34983a9c0ff22a30cec79" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33911 } } peers { permanent_uuid: "938ece79da564d42858dbda7e210b950" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33419 } } peers { permanent_uuid: "eaf2b4a72dd3425ea0889445dda7b897" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37153 } }
I20260928 14:40:25.848917 3372 raft_consensus.cc:374] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:25.848944 3372 raft_consensus.cc:729] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2de369bf2ce34983a9c0ff22a30cec79, State: Initialized, Role: FOLLOWER
I20260928 14:40:25.849056 3372 consensus_queue.cc:260] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de369bf2ce34983a9c0ff22a30cec79" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33911 } } peers { permanent_uuid: "938ece79da564d42858dbda7e210b950" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33419 } } peers { permanent_uuid: "eaf2b4a72dd3425ea0889445dda7b897" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37153 } }
I20260928 14:40:25.849318 3372 ts_tablet_manager.cc:1428] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:40:25.849448 3339 heartbeater.cc:503] Master 127.22.13.126:35737 was elected leader, sending a full tablet report...
W20260928 14:40:25.859364 3072 tablet.cc:2369] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:40:26.000871 3340 tablet.cc:2369] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 28 14:40:26 dist-test-slave-wwwb krb5kdc[1877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606425, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for HTTP/127.22.13.84@KRBTEST.COM
I20260928 14:40:26.043560 3379 raft_consensus.cc:482] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:40:26.043748 3379 raft_consensus.cc:504] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de369bf2ce34983a9c0ff22a30cec79" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33911 } } peers { permanent_uuid: "938ece79da564d42858dbda7e210b950" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33419 } } peers { permanent_uuid: "eaf2b4a72dd3425ea0889445dda7b897" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37153 } }
I20260928 14:40:26.044169 3379 leader_election.cc:288] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2de369bf2ce34983a9c0ff22a30cec79 (127.22.13.67:33911), 938ece79da564d42858dbda7e210b950 (127.22.13.66:33419)
I20260928 14:40:26.048213 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:26.044728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:57561 (local address 127.22.13.67:33911)
0928 14:40:26.044776 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:26.044778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:26.044786 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:40:26.044816 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:26.044817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:26.044826 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:26.044870 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:26.045055 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:26.045787 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:26.047207 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:26.047765 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:26.047815 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:26.047826 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:26.048076 (+ 250us) server_negotiation.cc:299] Negotiation successful
0928 14:40:26.048124 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 14:40:26.048486 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:26.044325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:33911 (local address 127.22.13.65:57561)
0928 14:40:26.044675 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:26.044689 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:26.044743 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:26.044884 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:26.044885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:26.044887 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:26.044969 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:26.044971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:26.045898 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:26.045899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:26.047087 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:26.047091 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:26.047912 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:26.047913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:26.047983 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:26.047987 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:40:26.048323 (+ 336us) client_negotiation.cc:240] Negotiation successful
0928 14:40:26.048386 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":111,"threads_started":1}
I20260928 14:40:26.048544 3370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:26.044747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:41241 (local address 127.22.13.66:33419)
0928 14:40:26.044790 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:26.044792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:26.044900 (+ 108us) server_negotiation.cc:407] Connection header received
0928 14:40:26.044998 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:26.044999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:26.045007 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:26.045035 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:26.045253 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:26.046036 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:26.047371 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:26.047974 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:26.048025 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:26.048034 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:26.048420 (+ 386us) server_negotiation.cc:299] Negotiation successful
0928 14:40:26.048469 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:40:26.048887 3382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:26.044547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:33419 (local address 127.22.13.65:41241)
0928 14:40:26.044842 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:26.044852 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:26.044925 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:26.045098 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:26.045098 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:26.045100 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:26.045172 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:26.045173 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:26.046135 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:26.046136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:26.047254 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:26.047258 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:26.048257 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:26.048258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:26.048333 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:26.048338 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:40:26.048759 (+ 421us) client_negotiation.cc:240] Negotiation successful
0928 14:40:26.048804 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":73,"threads_started":1}
I20260928 14:40:26.049006 3293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "628fce7b1fa3416180559820c01de0c5" candidate_uuid: "eaf2b4a72dd3425ea0889445dda7b897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2de369bf2ce34983a9c0ff22a30cec79" is_pre_election: true
I20260928 14:40:26.049191 3293 raft_consensus.cc:2454] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaf2b4a72dd3425ea0889445dda7b897 in term 0.
I20260928 14:40:26.049189 3159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "628fce7b1fa3416180559820c01de0c5" candidate_uuid: "eaf2b4a72dd3425ea0889445dda7b897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "938ece79da564d42858dbda7e210b950" is_pre_election: true
I20260928 14:40:26.049331 3159 raft_consensus.cc:2454] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaf2b4a72dd3425ea0889445dda7b897 in term 0.
I20260928 14:40:26.049444 2958 leader_election.cc:302] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2de369bf2ce34983a9c0ff22a30cec79, eaf2b4a72dd3425ea0889445dda7b897; no voters:
I20260928 14:40:26.049580 3379 raft_consensus.cc:2781] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:40:26.049664 3379 raft_consensus.cc:482] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:40:26.049700 3379 raft_consensus.cc:3037] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:26.050647 3379 raft_consensus.cc:504] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de369bf2ce34983a9c0ff22a30cec79" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33911 } } peers { permanent_uuid: "938ece79da564d42858dbda7e210b950" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33419 } } peers { permanent_uuid: "eaf2b4a72dd3425ea0889445dda7b897" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37153 } }
I20260928 14:40:26.050782 3379 leader_election.cc:288] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [CANDIDATE]: Term 1 election: Requested vote from peers 2de369bf2ce34983a9c0ff22a30cec79 (127.22.13.67:33911), 938ece79da564d42858dbda7e210b950 (127.22.13.66:33419)
I20260928 14:40:26.051051 3159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "628fce7b1fa3416180559820c01de0c5" candidate_uuid: "eaf2b4a72dd3425ea0889445dda7b897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "938ece79da564d42858dbda7e210b950"
I20260928 14:40:26.051106 3293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "628fce7b1fa3416180559820c01de0c5" candidate_uuid: "eaf2b4a72dd3425ea0889445dda7b897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2de369bf2ce34983a9c0ff22a30cec79"
I20260928 14:40:26.051129 3159 raft_consensus.cc:3037] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:26.051195 3293 raft_consensus.cc:3037] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:26.052016 3159 raft_consensus.cc:2454] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf2b4a72dd3425ea0889445dda7b897 in term 1.
I20260928 14:40:26.052147 3293 raft_consensus.cc:2454] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf2b4a72dd3425ea0889445dda7b897 in term 1.
I20260928 14:40:26.052342 2958 leader_election.cc:302] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 938ece79da564d42858dbda7e210b950, eaf2b4a72dd3425ea0889445dda7b897; no voters:
I20260928 14:40:26.052495 3379 raft_consensus.cc:2781] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:40:26.052682 3379 raft_consensus.cc:686] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [term 1 LEADER]: Becoming Leader. State: Replica: eaf2b4a72dd3425ea0889445dda7b897, State: Running, Role: LEADER
I20260928 14:40:26.052778 3379 consensus_queue.cc:237] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de369bf2ce34983a9c0ff22a30cec79" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33911 } } peers { permanent_uuid: "938ece79da564d42858dbda7e210b950" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33419 } } peers { permanent_uuid: "eaf2b4a72dd3425ea0889445dda7b897" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37153 } }
I20260928 14:40:26.054140 2872 catalog_manager.cc:5495] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 reported cstate change: term changed from 0 to 1, leader changed from <none> to eaf2b4a72dd3425ea0889445dda7b897 (127.22.13.65). New cstate: current_term: 1 leader_uuid: "eaf2b4a72dd3425ea0889445dda7b897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de369bf2ce34983a9c0ff22a30cec79" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "938ece79da564d42858dbda7e210b950" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf2b4a72dd3425ea0889445dda7b897" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37153 } health_report { overall_health: HEALTHY } } }
W20260928 14:40:26.064196 3206 tablet.cc:2369] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:40:26.103403 3368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:26.099967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56516 (local address 127.22.13.65:37153)
0928 14:40:26.100019 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:26.100021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:26.100028 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:40:26.100070 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:26.100071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:26.100095 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:26.100179 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:40:26.100408 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:26.100984 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:26.102196 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:26.102481 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:26.102559 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:26.102813 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:40:26.102955 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:40:26.103033 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:26.103293 (+ 260us) server_negotiation.cc:299] Negotiation successful
0928 14:40:26.103316 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 14:40:26.107203 3293 raft_consensus.cc:1261] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79 [term 1 FOLLOWER]: Refusing update from remote peer eaf2b4a72dd3425ea0889445dda7b897: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:40:26.107203 3159 raft_consensus.cc:1261] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950 [term 1 FOLLOWER]: Refusing update from remote peer eaf2b4a72dd3425ea0889445dda7b897: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:40:26.107642 3384 consensus_queue.cc:1035] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2de369bf2ce34983a9c0ff22a30cec79" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:40:26.107836 3379 consensus_queue.cc:1035] T 628fce7b1fa3416180559820c01de0c5 P eaf2b4a72dd3425ea0889445dda7b897 [LEADER]: Connected to new peer: Peer: permanent_uuid: "938ece79da564d42858dbda7e210b950" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:40:26.113179 3388 mvcc.cc:205] Tried to move back new op lower bound from 7334323921330458624 to 7334323921112801280. Current Snapshot: MvccSnapshot[applied={T|T < 7334323921330458624}]
I20260928 14:40:26.113459 3390 mvcc.cc:205] Tried to move back new op lower bound from 7334323921330458624 to 7334323921112801280. Current Snapshot: MvccSnapshot[applied={T|T < 7334323921330458624}]
I20260928 14:40:26.114149 3389 mvcc.cc:205] Tried to move back new op lower bound from 7334323921330458624 to 7334323921112801280. Current Snapshot: MvccSnapshot[applied={T|T < 7334323921330458624}]
I20260928 14:40:26.117048 2873 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:40:26.117152 2873 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:40:26.118052 2873 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 909a182c9df043a5b0c506cbbc4d6c63: Sending DeleteTablet for 3 replicas of tablet 628fce7b1fa3416180559820c01de0c5
I20260928 14:40:26.118813 3273 tablet_service.cc:1514] Processing DeleteTablet for tablet 628fce7b1fa3416180559820c01de0c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:40:26 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:54004
I20260928 14:40:26.119065 3139 tablet_service.cc:1514] Processing DeleteTablet for tablet 628fce7b1fa3416180559820c01de0c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:40:26 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:50224
I20260928 14:40:26.119350 3005 tablet_service.cc:1514] Processing DeleteTablet for tablet 628fce7b1fa3416180559820c01de0c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:40:26 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:56500
I20260928 14:40:26.119957 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 2911
I20260928 14:40:26.122649 3398 tablet_replica.cc:330] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79: stopping tablet replica
I20260928 14:40:26.122838 3399 tablet_replica.cc:330] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950: stopping tablet replica
I20260928 14:40:26.130158 3399 raft_consensus.cc:2229] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:40:26.130499 3398 raft_consensus.cc:2229] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:40:26.130823 3399 raft_consensus.cc:2258] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:40:26.131029 3398 raft_consensus.cc:2258] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:40:26.132156 3399 ts_tablet_manager.cc:1905] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:40:26.132300 3398 ts_tablet_manager.cc:1905] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:40:26.136600 3398 ts_tablet_manager.cc:1918] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:40:26.136720 3398 log.cc:1196] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/628fce7b1fa3416180559820c01de0c5
I20260928 14:40:26.137029 3398 ts_tablet_manager.cc:1939] T 628fce7b1fa3416180559820c01de0c5 P 2de369bf2ce34983a9c0ff22a30cec79: Deleting consensus metadata
I20260928 14:40:26.137588 2857 catalog_manager.cc:4841] TS 2de369bf2ce34983a9c0ff22a30cec79 (127.22.13.67:33911): tablet 628fce7b1fa3416180559820c01de0c5 (table test-table [id=212bc0d75f9740d2a827689206059b02]) successfully deleted
I20260928 14:40:26.138670 3399 ts_tablet_manager.cc:1918] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:40:26.138764 3399 log.cc:1196] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/628fce7b1fa3416180559820c01de0c5
I20260928 14:40:26.139042 3399 ts_tablet_manager.cc:1939] T 628fce7b1fa3416180559820c01de0c5 P 938ece79da564d42858dbda7e210b950: Deleting consensus metadata
W20260928 14:40:26.140695 2860 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:37153 (error 108)
I20260928 14:40:26.141110 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 3075
I20260928 14:40:26.141173 2857 catalog_manager.cc:4841] TS 938ece79da564d42858dbda7e210b950 (127.22.13.66:33419): tablet 628fce7b1fa3416180559820c01de0c5 (table test-table [id=212bc0d75f9740d2a827689206059b02]) successfully deleted
I20260928 14:40:26.150405 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:26.150259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:37153 (local address 127.0.0.1:56530)
0928 14:40:26.150317 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:26.150366 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 14:40:26.150683 2860 catalog_manager.cc:4584] TS eaf2b4a72dd3425ea0889445dda7b897 (127.22.13.65:37153): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 628fce7b1fa3416180559820c01de0c5: Network error: Client connection negotiation failed: client connection to 127.22.13.65:37153: connect: Connection refused (error 111)
I20260928 14:40:26.152156 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 3208
I20260928 14:40:26.160225 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 2840
I20260928 14:40:26.167514 22581 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260928 14:40:26.523214 22581 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260928 14:40:26.523319 22581 mini_ranger.cc:66] Stopping Ranger...
I20260928 14:40:26.868893 22581 mini_ranger.cc:68] Stopped Ranger
2026-09-28 14:40:26.869 UTC [1907] LOG: received smart shutdown request
2026-09-28 14:40:26.873 UTC [1907] LOG: background worker "logical replication launcher" (PID 1916) exited with exit code 1
2026-09-28 14:40:26.873 UTC [1911] LOG: shutting down
2026-09-28 14:40:27.042 UTC [1907] LOG: database system is shut down
2026-09-28T14:40:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50182 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:40:27 dist-test-slave-wwwb krb5kdc[3410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:40:27 dist-test-slave-wwwb krb5kdc[3410](info): set up 2 sockets
Sep 28 14:40:27 dist-test-slave-wwwb krb5kdc[3410](info): commencing operation
krb5kdc: starting...
W20260928 14:40:28.214089 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:40:28 dist-test-slave-wwwb krb5kdc[3410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:40:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:40:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:40:28.337177 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:46143
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:44929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:46143,127.22.13.125:34923,127.22.13.124:46163
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:28.422317 3423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:28.422609 3423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:28.422653 3423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:28.423964 3423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:40:28.424065 3423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:28.424090 3423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:28.424113 3423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:40:28.424142 3423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:40:28.425860 3423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:46143,127.22.13.125:34923,127.22.13.124:46163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:46143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:28.426223 3423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:28.426481 3423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:28.431651 3428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:28.431627 3429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:28.431789 3423 server_base.cc:1034] running on GCE node
W20260928 14:40:28.431617 3431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:28.432121 3423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:28.432547 3423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:28.433715 3423 hybrid_clock.cc:648] HybridClock initialized: now 1790606428433691 us; error 39 us; skew 500 ppm
Sep 28 14:40:28 dist-test-slave-wwwb krb5kdc[3410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606428, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:28.436474 3423 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:40:28.436995 3423 webserver.cc:460] Webserver started at http://127.22.13.126:41831/ using document root <none> and password file <none>
I20260928 14:40:28.437260 3423 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:28.437310 3423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:28.437484 3423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:40:28.438717 3423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "1beaa1821ae04440b0b5f55b96d5f8ba"
format_stamp: "Formatted at 2026-09-28 14:40:28 on dist-test-slave-wwwb"
server_key: "6050d6ba3c34fe6e77ac9609e74c1987"
server_key_iv: "dbbf8a96c53ca92bd12044d6df9bb7cc"
server_key_version: "encryptionkey@0"
I20260928 14:40:28.439118 3423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "1beaa1821ae04440b0b5f55b96d5f8ba"
format_stamp: "Formatted at 2026-09-28 14:40:28 on dist-test-slave-wwwb"
server_key: "6050d6ba3c34fe6e77ac9609e74c1987"
server_key_iv: "dbbf8a96c53ca92bd12044d6df9bb7cc"
server_key_version: "encryptionkey@0"
I20260928 14:40:28.440586 3423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:40:28.441385 3438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:28.441605 3423 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:28.441728 3423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "1beaa1821ae04440b0b5f55b96d5f8ba"
format_stamp: "Formatted at 2026-09-28 14:40:28 on dist-test-slave-wwwb"
server_key: "6050d6ba3c34fe6e77ac9609e74c1987"
server_key_iv: "dbbf8a96c53ca92bd12044d6df9bb7cc"
server_key_version: "encryptionkey@0"
I20260928 14:40:28.441841 3423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:28.451094 3423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:28.453217 3423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:28.453413 3423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:28.457792 3423 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:46143
I20260928 14:40:28.457849 3490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:46143 every 8 connection(s)
I20260928 14:40:28.458333 3423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:40:28.459311 3491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:28.460935 3491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:40:28.462285 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 3423
I20260928 14:40:28.462510 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:40:28.462702 22581 external_mini_cluster.cc:1428] Setting key 4a7afc90161ed4445d86bc23cd6633ad
Sep 28 14:40:28 dist-test-slave-wwwb krb5kdc[3410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606428, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.13.125@KRBTEST.COM; defaulting to no policy
I20260928 14:40:28.472126 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.461638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:46143 (local address 127.0.0.1:36402)
0928 14:40:28.461834 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.461878 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:28.462514 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:28.462873 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:28.462876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:28.463180 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:28.463358 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.463363 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.464427 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.464428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.465430 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.465434 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.466074 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.466076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.466173 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.466752 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:28.466765 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:28.468482 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:28.470477 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.470484 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.470487 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.470705 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.470971 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.470972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.470972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.471061 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.471449 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:28.471454 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:28.471521 (+ 67us) client_negotiation.cc:769] Sending connection context
0928 14:40:28.471633 (+ 112us) client_negotiation.cc:240] Negotiation successful
0928 14:40:28.471904 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":116,"thread_start_us":107,"threads_started":1}
I20260928 14:40:28.472126 3494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.461702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36402 (local address 127.22.13.126:46143)
0928 14:40:28.461874 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:28.461882 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:28.462007 (+ 125us) server_negotiation.cc:407] Connection header received
0928 14:40:28.462725 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:28.462727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:28.462751 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:28.462851 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:28.463485 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.464319 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.465533 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.465901 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.466077 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.468600 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:28.468608 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:28.468608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:28.468616 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:28.470366 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.470804 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.470808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.470808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.470898 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.471127 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.471128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.471129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.471373 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:28.471472 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:28.471659 (+ 187us) server_negotiation.cc:299] Negotiation successful
0928 14:40:28.471857 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.125@KRBTEST.COM" created.
I20260928 14:40:28.473120 3491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:40:28.473523 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.473426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:34923 (local address 127.0.0.1:38690)
0928 14:40:28.473461 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.473503 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:34923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260928 14:40:28.473695 3440 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.13.125:34923: connect: Connection refused (error 111)
I20260928 14:40:28.474171 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.474076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:34923 (local address 127.0.0.1:38694)
0928 14:40:28.474122 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.474147 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:34923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260928 14:40:28.474315 3491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.125:34923: Network error: Client connection negotiation failed: client connection to 127.22.13.125:34923: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.125.keytab.
Entry for principal kudu/127.22.13.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.125.keytab.
I20260928 14:40:28.488407 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.125:34923
--webserver_interface=127.22.13.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.125.keytab
--principal=kudu/127.22.13.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:44929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:46143,127.22.13.125:34923,127.22.13.124:46163
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:40:28.523592 3491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } attempt: 1
I20260928 14:40:28.524108 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.523971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:34923 (local address 127.0.0.1:38710)
0928 14:40:28.524038 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.524078 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:34923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260928 14:40:28.524459 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.524363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:34923 (local address 127.0.0.1:38716)
0928 14:40:28.524411 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.524437 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:34923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 14:40:28.524545 3491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.125:34923: Network error: Client connection negotiation failed: client connection to 127.22.13.125:34923: connect: Connection refused (error 111)
W20260928 14:40:28.571955 3498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:28.572260 3498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:28.572305 3498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:28.573575 3498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:40:28.573660 3498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:28.573693 3498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:28.573724 3498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:40:28.573755 3498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:40:28.575453 3498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:46143,127.22.13.125:34923,127.22.13.124:46163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.125.keytab
--principal=kudu/127.22.13.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.125:34923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.13.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:28.575793 3498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:28.576053 3498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:28.581125 3504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:28.581115 3506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:28.581151 3503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:28.581446 3498 server_base.cc:1034] running on GCE node
I20260928 14:40:28.581787 3498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:28.582203 3498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:28.583384 3498 hybrid_clock.cc:648] HybridClock initialized: now 1790606428583376 us; error 37 us; skew 500 ppm
Sep 28 14:40:28 dist-test-slave-wwwb krb5kdc[3410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606428, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:28.585920 3498 init.cc:377] Logged in from keytab as kudu/127.22.13.125@KRBTEST.COM (short username kudu)
I20260928 14:40:28.586519 3498 webserver.cc:460] Webserver started at http://127.22.13.125:44129/ using document root <none> and password file <none>
I20260928 14:40:28.586788 3498 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:28.586863 3498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:28.587033 3498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:40:28.588225 3498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/data/instance:
uuid: "3a069d6b853943f78008a5f8412609c2"
format_stamp: "Formatted at 2026-09-28 14:40:28 on dist-test-slave-wwwb"
server_key: "fe42384fdea1cab694d83ca6102a39b8"
server_key_iv: "56651d98db497223778bbfb454dc6266"
server_key_version: "encryptionkey@0"
I20260928 14:40:28.588608 3498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/wal/instance:
uuid: "3a069d6b853943f78008a5f8412609c2"
format_stamp: "Formatted at 2026-09-28 14:40:28 on dist-test-slave-wwwb"
server_key: "fe42384fdea1cab694d83ca6102a39b8"
server_key_iv: "56651d98db497223778bbfb454dc6266"
server_key_version: "encryptionkey@0"
I20260928 14:40:28.590102 3498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260928 14:40:28.590905 3513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:28.591086 3498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:28.591200 3498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/wal
uuid: "3a069d6b853943f78008a5f8412609c2"
format_stamp: "Formatted at 2026-09-28 14:40:28 on dist-test-slave-wwwb"
server_key: "fe42384fdea1cab694d83ca6102a39b8"
server_key_iv: "56651d98db497223778bbfb454dc6266"
server_key_version: "encryptionkey@0"
I20260928 14:40:28.591296 3498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:28.592682 3491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } attempt: 2
I20260928 14:40:28.593158 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.593005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:34923 (local address 127.0.0.1:38724)
0928 14:40:28.593075 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.593120 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:34923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260928 14:40:28.593573 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.593456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:34923 (local address 127.0.0.1:38730)
0928 14:40:28.593504 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.593533 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:34923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 14:40:28.593703 3491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.125:34923: Network error: Client connection negotiation failed: client connection to 127.22.13.125:34923: connect: Connection refused (error 111)
I20260928 14:40:28.610175 3498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:28.612147 3498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:28.612324 3498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:28.616667 3498 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.125:34923
I20260928 14:40:28.616708 3565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.125:34923 every 8 connection(s)
I20260928 14:40:28.617094 3498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/data/info.pb
I20260928 14:40:28.618088 3566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:28.619689 3566 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:40:28.623638 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 3498
I20260928 14:40:28.623759 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-1/wal/instance
I20260928 14:40:28.623962 22581 external_mini_cluster.cc:1428] Setting key d4681265f48be09cbef2168c3a001392
Sep 28 14:40:28 dist-test-slave-wwwb krb5kdc[3410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606428, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.125@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:28.629488 3494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.620440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36414 (local address 127.22.13.126:46143)
0928 14:40:28.620512 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:28.620514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:28.620702 (+ 188us) server_negotiation.cc:407] Connection header received
0928 14:40:28.621170 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:28.621170 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:28.621184 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:28.621251 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:28.621918 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.622459 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.623486 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.623826 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.623909 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.626280 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:28.626282 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:28.626282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:28.626289 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:28.627831 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.628401 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.628402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.628402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.628460 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.628793 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.628793 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.628794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.628905 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:28.628956 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:28.629342 (+ 386us) server_negotiation.cc:299] Negotiation successful
0928 14:40:28.629419 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:40:28.629864 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.620377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:46143 (local address 127.0.0.1:36414)
0928 14:40:28.620558 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.620603 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:28.621079 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:28.621332 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:28.621336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:28.621603 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:28.621829 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.621833 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.622555 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.622556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.623395 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.623398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.623916 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.623918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.624088 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.624650 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:28.624659 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:28.626150 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:28.627932 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.627944 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.627947 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.628314 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.628536 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.628537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.628537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.628720 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.628973 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:28.628981 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:28.629189 (+ 208us) client_negotiation.cc:769] Sending connection context
0928 14:40:28.629328 (+ 139us) client_negotiation.cc:240] Negotiation successful
0928 14:40:28.629543 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260928 14:40:28.630564 3566 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.13.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.124@KRBTEST.COM" created.
Sep 28 14:40:28 dist-test-slave-wwwb krb5kdc[3410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606428, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.125@KRBTEST.COM for kudu/127.22.13.125@KRBTEST.COM
I20260928 14:40:28.639036 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.630890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:34923 (local address 127.0.0.1:38738)
0928 14:40:28.631128 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.631140 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:28.631197 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:28.631352 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:28.631354 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:28.631509 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:28.631588 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.631591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.632523 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.632524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.633103 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.633105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.633789 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.633790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.633852 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.634353 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:28.634358 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:28.635681 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:28.637722 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.637724 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.637724 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.637997 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.638209 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.638209 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.638210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.638281 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.638571 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:28.638573 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:28.638629 (+ 56us) client_negotiation.cc:769] Sending connection context
0928 14:40:28.638725 (+ 96us) client_negotiation.cc:240] Negotiation successful
0928 14:40:28.638858 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220}
I20260928 14:40:28.639041 3570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.630963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38738 (local address 127.22.13.125:34923)
0928 14:40:28.631138 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:28.631143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:28.631199 (+ 56us) server_negotiation.cc:407] Connection header received
0928 14:40:28.631253 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:28.631255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:28.631267 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:28.631328 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:28.631667 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.632434 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.633275 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.633715 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.633779 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.635775 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:28.635788 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:28.635788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:28.635797 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:28.637615 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.638077 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.638080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.638081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.638137 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.638342 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.638343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.638343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.638499 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:28.638561 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:28.638779 (+ 218us) server_negotiation.cc:299] Negotiation successful
0928 14:40:28.638924 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
I20260928 14:40:28.639844 3566 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:40:28.640234 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.640081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:46163 (local address 127.0.0.1:54316)
0928 14:40:28.640138 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.640196 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260928 14:40:28.640398 3514 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.13.124:46163: connect: Connection refused (error 111)
I20260928 14:40:28.640880 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.640767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:46163 (local address 127.0.0.1:54324)
0928 14:40:28.640818 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.640849 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 14:40:28.641041 3566 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.124:46163: Network error: Client connection negotiation failed: client connection to 127.22.13.124:46163: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.124.keytab.
Entry for principal kudu/127.22.13.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.124.keytab.
I20260928 14:40:28.650933 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.124:46163
--webserver_interface=127.22.13.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.124.keytab
--principal=kudu/127.22.13.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:44929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:46143,127.22.13.125:34923,127.22.13.124:46163
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:40:28.684887 3491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } attempt: 3
Sep 28 14:40:28 dist-test-slave-wwwb krb5kdc[3410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606428, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for kudu/127.22.13.125@KRBTEST.COM
I20260928 14:40:28.690284 3566 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } attempt: 1
I20260928 14:40:28.690673 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.690554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:46163 (local address 127.0.0.1:54340)
0928 14:40:28.690613 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.690640 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260928 14:40:28.691036 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.690915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:46163 (local address 127.0.0.1:54344)
0928 14:40:28.690972 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.691005 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 14:40:28.691315 3566 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.124:46163: Network error: Client connection negotiation failed: client connection to 127.22.13.124:46163: connect: Connection refused (error 111)
I20260928 14:40:28.692963 3570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.685424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38744 (local address 127.22.13.125:34923)
0928 14:40:28.685488 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:28.685490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:28.685526 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:40:28.685552 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:28.685553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:28.685564 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:28.685609 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:28.686068 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.686587 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.687644 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.687855 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.687909 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.690157 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:28.690159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:28.690159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:28.690167 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:28.691792 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.692161 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.692163 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.692163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.692227 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.692428 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.692429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.692429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.692552 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:28.692626 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:28.692791 (+ 165us) server_negotiation.cc:299] Negotiation successful
0928 14:40:28.692873 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 14:40:28.692970 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.685344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:34923 (local address 127.0.0.1:38744)
0928 14:40:28.685413 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.685425 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:28.685470 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:28.685646 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:28.685647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:28.685861 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:28.685982 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.685984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.686691 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.686693 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.687536 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.687540 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.687942 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.687943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.688048 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.688522 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:28.688529 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:28.690046 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:28.691882 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.691883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.691884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.692073 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.692295 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.692296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.692296 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.692362 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.692616 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:28.692617 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:28.692668 (+ 51us) client_negotiation.cc:769] Sending connection context
0928 14:40:28.692767 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 14:40:28.692873 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260928 14:40:28.693401 3491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:40:28.693758 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.693652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:46163 (local address 127.0.0.1:54354)
0928 14:40:28.693704 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.693731 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260928 14:40:28.694164 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.694090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:46163 (local address 127.0.0.1:54368)
0928 14:40:28.694127 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.694149 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260928 14:40:28.694267 3491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.124:46163: Network error: Client connection negotiation failed: client connection to 127.22.13.124:46163: connect: Connection refused (error 111)
I20260928 14:40:28.725430 3491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } attempt: 1
I20260928 14:40:28.726058 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.725865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:46163 (local address 127.0.0.1:54382)
0928 14:40:28.725979 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.726030 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260928 14:40:28.726441 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.726360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:46163 (local address 127.0.0.1:54394)
0928 14:40:28.726401 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.726426 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260928 14:40:28.726558 3491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.124:46163: Network error: Client connection negotiation failed: client connection to 127.22.13.124:46163: connect: Connection refused (error 111)
W20260928 14:40:28.737277 3573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:28.737558 3573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:28.737617 3573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:28.738945 3573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:40:28.739028 3573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:28.739053 3573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:28.739084 3573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:40:28.739106 3573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:40:28.740725 3573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:46143,127.22.13.125:34923,127.22.13.124:46163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.124.keytab
--principal=kudu/127.22.13.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.124:46163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.13.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:28.741037 3573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:28.741302 3573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:28.746331 3578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:28.746367 3581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:28.746393 3579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:28.746779 3573 server_base.cc:1034] running on GCE node
I20260928 14:40:28.747046 3573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:28.747371 3573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:28.748550 3573 hybrid_clock.cc:648] HybridClock initialized: now 1790606428748537 us; error 44 us; skew 500 ppm
Sep 28 14:40:28 dist-test-slave-wwwb krb5kdc[3410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606428, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:28.751184 3573 init.cc:377] Logged in from keytab as kudu/127.22.13.124@KRBTEST.COM (short username kudu)
I20260928 14:40:28.751670 3573 webserver.cc:460] Webserver started at http://127.22.13.124:44793/ using document root <none> and password file <none>
I20260928 14:40:28.751909 3573 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:28.751979 3573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:28.752134 3573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:40:28.753261 3573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/data/instance:
uuid: "08e50d13a8a146acb1cd37e919d3ce95"
format_stamp: "Formatted at 2026-09-28 14:40:28 on dist-test-slave-wwwb"
server_key: "aece6add7a36b7c7a55f417a540d93fe"
server_key_iv: "d344754a1a9442f07dea2be8904d9937"
server_key_version: "encryptionkey@0"
I20260928 14:40:28.753654 3573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/wal/instance:
uuid: "08e50d13a8a146acb1cd37e919d3ce95"
format_stamp: "Formatted at 2026-09-28 14:40:28 on dist-test-slave-wwwb"
server_key: "aece6add7a36b7c7a55f417a540d93fe"
server_key_iv: "d344754a1a9442f07dea2be8904d9937"
server_key_version: "encryptionkey@0"
I20260928 14:40:28.755262 3573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:40:28.756179 3588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:28.756383 3573 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:40:28.756494 3573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/wal
uuid: "08e50d13a8a146acb1cd37e919d3ce95"
format_stamp: "Formatted at 2026-09-28 14:40:28 on dist-test-slave-wwwb"
server_key: "aece6add7a36b7c7a55f417a540d93fe"
server_key_iv: "d344754a1a9442f07dea2be8904d9937"
server_key_version: "encryptionkey@0"
I20260928 14:40:28.756578 3573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:28.759472 3566 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } attempt: 2
I20260928 14:40:28.759915 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.759781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:46163 (local address 127.0.0.1:54406)
0928 14:40:28.759844 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.759885 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260928 14:40:28.760265 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.760164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:46163 (local address 127.0.0.1:54414)
0928 14:40:28.760212 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.760245 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 14:40:28.760373 3566 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.124:46163: Network error: Client connection negotiation failed: client connection to 127.22.13.124:46163: connect: Connection refused (error 111)
I20260928 14:40:28.769477 3573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:28.771595 3573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:28.771729 3573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:28.776039 3573 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.124:46163
I20260928 14:40:28.776075 3640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.124:46163 every 8 connection(s)
I20260928 14:40:28.776476 3573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/data/info.pb
I20260928 14:40:28.777397 3641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:28.779129 3641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:40:28 dist-test-slave-wwwb krb5kdc[3410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606428, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.124@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:28.786512 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 3573
I20260928 14:40:28.786645 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606270465379-22581-0/minicluster-data/master-2/wal/instance
I20260928 14:40:28.786856 22581 external_mini_cluster.cc:1428] Setting key 84e440f7501c9ded8f756b507e27b9d4
I20260928 14:40:28.788846 3494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.779870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36422 (local address 127.22.13.126:46143)
0928 14:40:28.779965 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:28.779967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:28.780145 (+ 178us) server_negotiation.cc:407] Connection header received
0928 14:40:28.780621 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:28.780622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:28.780632 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:28.780697 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:28.781328 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.781807 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.782897 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.783089 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.783146 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.785548 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:28.785550 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:28.785550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:28.785557 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:28.787160 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.787759 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.787760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.787761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.787821 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.788135 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.788135 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.788136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.788250 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:28.788300 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:28.788648 (+ 348us) server_negotiation.cc:299] Negotiation successful
0928 14:40:28.788742 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 14:40:28.788936 3643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.779810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:46143 (local address 127.0.0.1:36422)
0928 14:40:28.780017 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.780057 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:28.780524 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:28.780763 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:28.780767 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:28.781030 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:28.781249 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.781254 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.781896 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.781897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.782812 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.782814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.783169 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.783170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.783284 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.783854 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:28.783862 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:28.785446 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:28.787257 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.787265 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.787268 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.787663 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.787896 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.787897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.787898 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.788060 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.788323 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:28.788330 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:28.788511 (+ 181us) client_negotiation.cc:769] Sending connection context
0928 14:40:28.788601 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 14:40:28.788735 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":118,"threads_started":1}
I20260928 14:40:28.789559 3641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:40:28 dist-test-slave-wwwb krb5kdc[3410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.124@KRBTEST.COM
Sep 28 14:40:28 dist-test-slave-wwwb krb5kdc[3410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606428, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.124@KRBTEST.COM for kudu/127.22.13.125@KRBTEST.COM
I20260928 14:40:28.797878 3643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.789864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:34923 (local address 127.0.0.1:38756)
0928 14:40:28.789920 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.789932 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:28.789993 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:28.790144 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:28.790145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:28.790308 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:28.790371 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.790374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.791106 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.791107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.791771 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.791793 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.792171 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.792173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.792255 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.792593 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:28.792599 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:28.794743 (+ 2144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:28.796650 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.796651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.796652 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.796889 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.797101 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.797102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.797103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.797174 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.797478 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:28.797480 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:28.797572 (+ 92us) client_negotiation.cc:769] Sending connection context
0928 14:40:28.797669 (+ 97us) client_negotiation.cc:240] Negotiation successful
0928 14:40:28.797804 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260928 14:40:28.797892 3570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.789967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38756 (local address 127.22.13.125:34923)
0928 14:40:28.790005 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:28.790006 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:28.790014 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:40:28.790070 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:28.790070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:28.790079 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:28.790120 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:28.790450 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.791007 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.791890 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.792095 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.792151 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.794846 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:28.794847 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:28.794847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:28.794855 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:28.796533 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.796966 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.796967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.796967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.797032 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.797248 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.797248 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.797249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.797385 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:28.797456 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:28.797716 (+ 260us) server_negotiation.cc:299] Negotiation successful
0928 14:40:28.797807 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260928 14:40:28.798302 3641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:40:28.798730 3645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.787272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54420 (local address 127.22.13.124:46163)
0928 14:40:28.787444 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:28.787457 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:28.787493 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:40:28.787571 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:28.787575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:28.787587 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:28.787654 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:28.788415 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.789563 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.790802 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.791273 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.791329 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.794912 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:28.794927 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:28.794927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:28.794934 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:28.796976 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.797420 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.797425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.797425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.797486 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.797735 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.797736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.797736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.798011 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:28.798202 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:28.798433 (+ 231us) server_negotiation.cc:299] Negotiation successful
0928 14:40:28.798516 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":82,"threads_started":1}
I20260928 14:40:28.801709 3491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } attempt: 2
Sep 28 14:40:28 dist-test-slave-wwwb krb5kdc[3410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606428, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.124@KRBTEST.COM for kudu/127.22.13.124@KRBTEST.COM
Sep 28 14:40:28 dist-test-slave-wwwb krb5kdc[3410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
Sep 28 14:40:28 dist-test-slave-wwwb krb5kdc[3410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606428, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for kudu/127.22.13.124@KRBTEST.COM
I20260928 14:40:28.807296 3643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.798603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:46163 (local address 127.0.0.1:54434)
0928 14:40:28.798642 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.798655 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:28.798692 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:28.798953 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:28.798953 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:28.799108 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:28.799140 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.799141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.800202 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.800203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.800678 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.800679 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.801086 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.801086 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.801134 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.801602 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:28.801607 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:28.803124 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:28.806130 (+ 3006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.806131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.806132 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.806347 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.806587 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.806588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.806589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.806661 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.806949 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:28.806950 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:28.806989 (+ 39us) client_negotiation.cc:769] Sending connection context
0928 14:40:28.807058 (+ 69us) client_negotiation.cc:240] Negotiation successful
0928 14:40:28.807165 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260928 14:40:28.807616 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.798650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54434 (local address 127.22.13.124:46163)
0928 14:40:28.798824 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:28.798826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:28.798834 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:40:28.798855 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:28.798856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:28.798869 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:28.798953 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:28.799251 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.800105 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.800779 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.800997 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.801086 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.803208 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:28.803210 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:28.803210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:28.803217 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:28.806038 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.806434 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.806435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.806436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.806500 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.806731 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.806731 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.806732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.806864 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:28.806927 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:28.807103 (+ 176us) server_negotiation.cc:299] Negotiation successful
0928 14:40:28.807532 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":334,"server-negotiator.queue_time_us":154,"thread_start_us":128,"threads_started":1}
I20260928 14:40:28.808187 3494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.799569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36428 (local address 127.22.13.126:46143)
0928 14:40:28.799609 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:28.799611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:28.799619 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:40:28.799644 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:28.799645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:28.799660 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:28.799706 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:28.800264 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.800723 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.801427 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.801641 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.801746 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.804553 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:28.804555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:28.804555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:28.804563 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:28.806612 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.807006 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.807007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.807008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.807069 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.807311 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.807312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.807313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.807678 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:28.807885 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:28.807992 (+ 107us) server_negotiation.cc:299] Negotiation successful
0928 14:40:28.808077 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:40:28.809314 3641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95: Bootstrap starting.
I20260928 14:40:28.809793 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.802098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:46163 (local address 127.0.0.1:54436)
0928 14:40:28.802152 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.802163 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:28.802218 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:28.802346 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:28.802347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:28.802516 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:28.802566 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.802567 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.803483 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.803484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.804085 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.804087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.804496 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.804497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.804555 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.804968 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:28.804973 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:28.806610 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:28.808690 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.808691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.808692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.808869 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.809093 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.809094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.809095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.809154 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.809464 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:28.809466 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:28.809511 (+ 45us) client_negotiation.cc:769] Sending connection context
0928 14:40:28.809571 (+ 60us) client_negotiation.cc:240] Negotiation successful
0928 14:40:28.809670 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 14:40:28.810181 3645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.802024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54436 (local address 127.22.13.124:46163)
0928 14:40:28.802097 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:28.802099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:28.802220 (+ 121us) server_negotiation.cc:407] Connection header received
0928 14:40:28.802270 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:28.802270 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:28.802284 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:28.802343 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:28.802652 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.803396 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.804192 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.804419 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.804496 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.806682 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:28.806683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:28.806683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:28.806689 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:28.808590 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.808952 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.808953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.808953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.809015 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.809213 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.809215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.809215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.809373 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:28.809905 (+ 532us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:28.809974 (+ 69us) server_negotiation.cc:299] Negotiation successful
0928 14:40:28.810060 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 14:40:28.810180 3641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:28.810839 3641 log.cc:824] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:28.811690 3641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95: No bootstrap required, opened a new log
I20260928 14:40:28.811873 3491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba: Bootstrap starting.
I20260928 14:40:28.812593 3491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:28.812927 3491 log.cc:824] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:28.813889 3491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba: No bootstrap required, opened a new log
Sep 28 14:40:28 dist-test-slave-wwwb krb5kdc[3410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.125@KRBTEST.COM
I20260928 14:40:28.813900 3641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } }
I20260928 14:40:28.814131 3641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:28.814169 3641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e50d13a8a146acb1cd37e919d3ce95, State: Initialized, Role: FOLLOWER
I20260928 14:40:28.814338 3641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } }
I20260928 14:40:28.814682 3641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:40:28.816443 3491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } }
I20260928 14:40:28.816609 3491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:28.816671 3491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1beaa1821ae04440b0b5f55b96d5f8ba, State: Initialized, Role: FOLLOWER
I20260928 14:40:28.816805 3491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } }
I20260928 14:40:28.817159 3491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba [sys.catalog]: configured and running, proceeding with master startup.
W20260928 14:40:28.817905 3660 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:40:28.818054 3660 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:40:28.818186 3570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.809445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38760 (local address 127.22.13.125:34923)
0928 14:40:28.809489 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:28.809491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:28.809510 (+ 19us) server_negotiation.cc:407] Connection header received
0928 14:40:28.809535 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:28.809536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:28.809545 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:28.809586 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:28.810397 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.811123 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.811796 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.812008 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.812101 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.814686 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:28.814687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:28.814688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:28.814696 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:28.816567 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.817003 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.817005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.817005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.817079 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.817323 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.817324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.817325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.817484 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:28.817701 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:28.817986 (+ 285us) server_negotiation.cc:299] Negotiation successful
0928 14:40:28.818078 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:40:28.819222 3661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } } }
I20260928 14:40:28.819423 3661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:40:28.819588 3649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } } }
I20260928 14:40:28.819937 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260928 14:40:28.820139 3673 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:40:28.820188 3673 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:40:28.816964 3573 master_runner.cc:186] Error getting master registration for 127.22.13.126:46143: OK, instance_id { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" instance_seqno: 1790606428456509 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:40:28.821583 3573 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:40:28.821641 3649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:40:28.822036 3423 master_runner.cc:186] Error getting master registration for 127.22.13.125:34923: OK, instance_id { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" instance_seqno: 1790606428615420 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:40:28.824494 3423 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:40:28.829833 3494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.821447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36436 (local address 127.22.13.126:46143)
0928 14:40:28.821499 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:28.821501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:28.822055 (+ 554us) server_negotiation.cc:407] Connection header received
0928 14:40:28.822154 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:28.822155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:28.822164 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:28.822194 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:28.822960 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.823682 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.824683 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.824971 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.825016 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.826674 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:28.826676 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:28.826676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:28.826684 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:28.828407 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.828889 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.828891 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.828891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.828954 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.829187 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.829187 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.829188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.829320 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:28.829397 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:28.829615 (+ 218us) server_negotiation.cc:299] Negotiation successful
0928 14:40:28.829700 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:40:28.833403 3570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.821777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38772 (local address 127.22.13.125:34923)
0928 14:40:28.821821 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:28.821823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:28.821862 (+ 39us) server_negotiation.cc:407] Connection header received
0928 14:40:28.824712 (+ 2850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:28.824713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:28.824725 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:28.824775 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:28.825515 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.826330 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.827349 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.827685 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.827752 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.829421 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:28.829423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:28.829423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:28.829431 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:28.831938 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.832297 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.832298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.832299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.832355 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.832532 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.832532 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.832533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.832660 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:28.832759 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:28.833198 (+ 439us) server_negotiation.cc:299] Negotiation successful
0928 14:40:28.833287 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:40:28.837103 3645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.822107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54444 (local address 127.22.13.124:46163)
0928 14:40:28.824585 (+ 2478us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:28.824587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:28.830289 (+ 5702us) server_negotiation.cc:407] Connection header received
0928 14:40:28.830402 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:28.830404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:28.830416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:28.830444 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:28.831002 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.831810 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.832819 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.833135 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.833177 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.834546 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:28.834547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:28.834548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:28.834556 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:28.835967 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.836310 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.836311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.836312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.836368 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.836584 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.836584 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.836585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.836697 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:28.836808 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:28.836923 (+ 115us) server_negotiation.cc:299] Negotiation successful
0928 14:40:28.836998 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2453}
I20260928 14:40:28.851514 3566 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } attempt: 3
Sep 28 14:40:28 dist-test-slave-wwwb krb5kdc[3410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606428, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.125@KRBTEST.COM for kudu/127.22.13.124@KRBTEST.COM
I20260928 14:40:28.859144 3645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.851913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54458 (local address 127.22.13.124:46163)
0928 14:40:28.851954 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:28.851956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:28.851966 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:40:28.852017 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:28.852018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:28.852029 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:28.852080 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:28.852391 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.852883 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.853594 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.853788 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.853863 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.855857 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:28.855859 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:28.855859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:28.855867 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:28.857877 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.858308 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.858309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.858310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.858372 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:28.858578 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:28.858579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:28.858579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:28.858706 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:28.858791 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:28.858960 (+ 169us) server_negotiation.cc:299] Negotiation successful
0928 14:40:28.859040 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:40:28.859144 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:28.851845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:46163 (local address 127.0.0.1:54458)
0928 14:40:28.851897 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:28.851911 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:28.851956 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:28.852093 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:28.852093 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:28.852260 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:28.852308 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.852310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.852978 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.852979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.853496 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:28.853498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:28.853862 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:28.853863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:28.853979 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:28.854456 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:28.854468 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:28.855761 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:28.858032 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.858033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.858033 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.858210 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.858439 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:28.858440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:28.858440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:28.858504 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:28.858783 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:28.858784 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:28.858835 (+ 51us) client_negotiation.cc:769] Sending connection context
0928 14:40:28.858918 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 14:40:28.859043 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260928 14:40:28.861171 3566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2: Bootstrap starting.
I20260928 14:40:28.861835 3566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:28.862190 3566 log.cc:824] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:28.863037 3566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2: No bootstrap required, opened a new log
I20260928 14:40:28.865393 3566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } }
I20260928 14:40:28.865541 3566 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:28.865605 3566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a069d6b853943f78008a5f8412609c2, State: Initialized, Role: FOLLOWER
I20260928 14:40:28.865763 3566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } }
I20260928 14:40:28.866078 3683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } } }
I20260928 14:40:28.866158 3566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:40:28.866222 3683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2 [sys.catalog]: This master's current role is: FOLLOWER
W20260928 14:40:28.868070 3694 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:40:28.868184 3694 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:40:28.870544 3498 master_runner.cc:263] No leader master found from master 3a069d6b853943f78008a5f8412609c2
I20260928 14:40:28.870616 3498 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:40:28.900835 3649 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:40:28.901010 3649 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } }
I20260928 14:40:28.901329 3649 leader_election.cc:288] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1beaa1821ae04440b0b5f55b96d5f8ba (127.22.13.126:46143), 3a069d6b853943f78008a5f8412609c2 (127.22.13.125:34923)
I20260928 14:40:28.901908 3540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08e50d13a8a146acb1cd37e919d3ce95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a069d6b853943f78008a5f8412609c2" is_pre_election: true
I20260928 14:40:28.902137 3540 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08e50d13a8a146acb1cd37e919d3ce95 in term 0.
I20260928 14:40:28.902511 3590 leader_election.cc:302] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08e50d13a8a146acb1cd37e919d3ce95, 3a069d6b853943f78008a5f8412609c2; no voters:
I20260928 14:40:28.902983 3465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08e50d13a8a146acb1cd37e919d3ce95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" is_pre_election: true
I20260928 14:40:28.903139 3465 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08e50d13a8a146acb1cd37e919d3ce95 in term 0.
I20260928 14:40:28.903497 3649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:40:28.903553 3649 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:40:28.903577 3649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:28.904560 3649 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } }
I20260928 14:40:28.904691 3649 leader_election.cc:288] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [CANDIDATE]: Term 1 election: Requested vote from peers 1beaa1821ae04440b0b5f55b96d5f8ba (127.22.13.126:46143), 3a069d6b853943f78008a5f8412609c2 (127.22.13.125:34923)
I20260928 14:40:28.905042 3540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08e50d13a8a146acb1cd37e919d3ce95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a069d6b853943f78008a5f8412609c2"
I20260928 14:40:28.905131 3540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:28.906136 3540 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08e50d13a8a146acb1cd37e919d3ce95 in term 1.
I20260928 14:40:28.906383 3590 leader_election.cc:302] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08e50d13a8a146acb1cd37e919d3ce95, 3a069d6b853943f78008a5f8412609c2; no voters:
I20260928 14:40:28.906602 3465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "08e50d13a8a146acb1cd37e919d3ce95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba"
I20260928 14:40:28.906706 3465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:28.907619 3465 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08e50d13a8a146acb1cd37e919d3ce95 in term 1.
I20260928 14:40:28.908880 3649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:40:28.909057 3649 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [term 1 LEADER]: Becoming Leader. State: Replica: 08e50d13a8a146acb1cd37e919d3ce95, State: Running, Role: LEADER
I20260928 14:40:28.909157 3649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } }
I20260928 14:40:28.909574 3649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08e50d13a8a146acb1cd37e919d3ce95. Latest consensus state: current_term: 1 leader_uuid: "08e50d13a8a146acb1cd37e919d3ce95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } } }
I20260928 14:40:28.909642 3649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:28.910005 3698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:40:28.910251 3698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:40:28.912541 3540 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2 [term 1 FOLLOWER]: Refusing update from remote peer 08e50d13a8a146acb1cd37e919d3ce95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:40:28.912827 3465 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba [term 1 FOLLOWER]: Refusing update from remote peer 08e50d13a8a146acb1cd37e919d3ce95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:40:28.913367 3649 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:40:28.913519 3649 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:40:28.914990 3661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08e50d13a8a146acb1cd37e919d3ce95. Latest consensus state: current_term: 1 leader_uuid: "08e50d13a8a146acb1cd37e919d3ce95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } } }
I20260928 14:40:28.915091 3661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:40:28.916308 3683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08e50d13a8a146acb1cd37e919d3ce95. Latest consensus state: current_term: 1 leader_uuid: "08e50d13a8a146acb1cd37e919d3ce95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } } }
I20260928 14:40:28.916405 3683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:40:28.917831 3649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "08e50d13a8a146acb1cd37e919d3ce95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } } }
I20260928 14:40:28.917990 3649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:28.918458 3661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "08e50d13a8a146acb1cd37e919d3ce95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } } }
I20260928 14:40:28.918546 3661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:40:28.918898 3699 mvcc.cc:205] Tried to move back new op lower bound from 7334323932822654976 to 7334323932812636160. Current Snapshot: MvccSnapshot[applied={T|T < 7334323932822654976}]
I20260928 14:40:28.920517 3700 mvcc.cc:205] Tried to move back new op lower bound from 7334323932822654976 to 7334323932812636160. Current Snapshot: MvccSnapshot[applied={T|T < 7334323932822654976}]
I20260928 14:40:28.921592 3696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "08e50d13a8a146acb1cd37e919d3ce95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } } }
I20260928 14:40:28.921707 3696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95 [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:28.922313 3683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "08e50d13a8a146acb1cd37e919d3ce95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1beaa1821ae04440b0b5f55b96d5f8ba" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46143 } } peers { permanent_uuid: "3a069d6b853943f78008a5f8412609c2" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 34923 } } peers { permanent_uuid: "08e50d13a8a146acb1cd37e919d3ce95" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 46163 } } }
I20260928 14:40:28.922405 3683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:40:28.923231 3702 mvcc.cc:205] Tried to move back new op lower bound from 7334323932822654976 to 7334323932812636160. Current Snapshot: MvccSnapshot[applied={T|T < 7334323932822654976}]
I20260928 14:40:28.924120 3698 catalog_manager.cc:1347] Generated new cluster ID: 656119a1bda0452287caba819809badf
I20260928 14:40:28.924170 3698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:40:28.941332 3698 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:40:28.942235 3698 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:40:28.953183 3698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08e50d13a8a146acb1cd37e919d3ce95: Generated new TSK 0
I20260928 14:40:28.953567 3698 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:40:29.820632 3673 catalog_manager.cc:1259] Loaded cluster ID: 656119a1bda0452287caba819809badf
I20260928 14:40:29.820737 3673 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba: loading cluster ID for follower catalog manager: success
I20260928 14:40:29.821717 3673 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba: acquiring CA information for follower catalog manager: success
I20260928 14:40:29.822104 3673 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1beaa1821ae04440b0b5f55b96d5f8ba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:40:29.868654 3694 catalog_manager.cc:1259] Loaded cluster ID: 656119a1bda0452287caba819809badf
I20260928 14:40:29.868738 3694 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2: loading cluster ID for follower catalog manager: success
I20260928 14:40:29.869572 3694 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2: acquiring CA information for follower catalog manager: success
I20260928 14:40:29.869915 3694 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3a069d6b853943f78008a5f8412609c2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:40:30.016944 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 3423
I20260928 14:40:30.022325 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 3498
I20260928 14:40:30.027566 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 3573
2026-09-28T14:40:30Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2869 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:40:30 dist-test-slave-wwwb krb5kdc[3714](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:40:30 dist-test-slave-wwwb krb5kdc[3714](info): set up 2 sockets
Sep 28 14:40:30 dist-test-slave-wwwb krb5kdc[3714](info): commencing operation
krb5kdc: starting...
W20260928 14:40:31.062438 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:40:31 dist-test-slave-wwwb krb5kdc[3714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:40:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:40:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:40:31.182942 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:45647
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:36905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:45647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:31.267010 3727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:31.267284 3727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:31.267326 3727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:31.268612 3727 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:40:31.268707 3727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:31.268733 3727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:31.268769 3727 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:40:31.268816 3727 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:40:31.270591 3727 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:36905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:45647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:45647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:31.270918 3727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:31.271170 3727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:31.276284 3735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:31.276263 3733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:31.276285 3732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:31.276840 3727 server_base.cc:1034] running on GCE node
I20260928 14:40:31.277123 3727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:31.277446 3727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:31.278626 3727 hybrid_clock.cc:648] HybridClock initialized: now 1790606431278609 us; error 55 us; skew 500 ppm
Sep 28 14:40:31 dist-test-slave-wwwb krb5kdc[3714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606431, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:31.281306 3727 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:40:31.281817 3727 webserver.cc:460] Webserver started at http://127.22.13.126:42215/ using document root <none> and password file <none>
I20260928 14:40:31.282092 3727 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:31.282145 3727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:31.282253 3727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:40:31.283438 3727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "6954c779ad8b42ef9223c0dc56ea5f5b"
format_stamp: "Formatted at 2026-09-28 14:40:31 on dist-test-slave-wwwb"
server_key: "0a5d335852d5f8a5b79e44a743891d53"
server_key_iv: "8edcc80e2fdd4b78fcabfb8bc42a51c2"
server_key_version: "encryptionkey@0"
I20260928 14:40:31.283772 3727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "6954c779ad8b42ef9223c0dc56ea5f5b"
format_stamp: "Formatted at 2026-09-28 14:40:31 on dist-test-slave-wwwb"
server_key: "0a5d335852d5f8a5b79e44a743891d53"
server_key_iv: "8edcc80e2fdd4b78fcabfb8bc42a51c2"
server_key_version: "encryptionkey@0"
I20260928 14:40:31.285176 3727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:40:31.286069 3742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:31.286355 3727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:31.286454 3727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "6954c779ad8b42ef9223c0dc56ea5f5b"
format_stamp: "Formatted at 2026-09-28 14:40:31 on dist-test-slave-wwwb"
server_key: "0a5d335852d5f8a5b79e44a743891d53"
server_key_iv: "8edcc80e2fdd4b78fcabfb8bc42a51c2"
server_key_version: "encryptionkey@0"
I20260928 14:40:31.286515 3727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:31.300115 3727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:31.302316 3727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:31.302490 3727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:31.306720 3727 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:45647
I20260928 14:40:31.306782 3794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:45647 every 8 connection(s)
I20260928 14:40:31.307178 3727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:40:31.308197 3795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:31.308408 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 3727
I20260928 14:40:31.308508 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:40:31.308734 22581 external_mini_cluster.cc:1428] Setting key 2077197278ffd28f9db46e8d69a33779
I20260928 14:40:31.312050 3795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b: Bootstrap starting.
I20260928 14:40:31.312819 3795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:31.313146 3795 log.cc:824] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:31.314025 3795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b: No bootstrap required, opened a new log
Sep 28 14:40:31 dist-test-slave-wwwb krb5kdc[3714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:31.316218 3795 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6954c779ad8b42ef9223c0dc56ea5f5b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 45647 } }
I20260928 14:40:31.316437 3795 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:31.316486 3795 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6954c779ad8b42ef9223c0dc56ea5f5b, State: Initialized, Role: FOLLOWER
I20260928 14:40:31.316635 3795 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6954c779ad8b42ef9223c0dc56ea5f5b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 45647 } }
I20260928 14:40:31.316720 3795 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:40:31.316764 3795 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:40:31.316860 3795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:31.317723 3795 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6954c779ad8b42ef9223c0dc56ea5f5b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 45647 } }
I20260928 14:40:31.317852 3795 leader_election.cc:302] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6954c779ad8b42ef9223c0dc56ea5f5b; no voters:
I20260928 14:40:31.318084 3795 leader_election.cc:288] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:40:31.318181 3800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:40:31.318403 3800 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [term 1 LEADER]: Becoming Leader. State: Replica: 6954c779ad8b42ef9223c0dc56ea5f5b, State: Running, Role: LEADER
I20260928 14:40:31.318400 3795 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:40:31.318583 3800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6954c779ad8b42ef9223c0dc56ea5f5b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 45647 } }
I20260928 14:40:31.318966 3801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6954c779ad8b42ef9223c0dc56ea5f5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6954c779ad8b42ef9223c0dc56ea5f5b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 45647 } } }
I20260928 14:40:31.319051 3801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:31.319226 3802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6954c779ad8b42ef9223c0dc56ea5f5b. Latest consensus state: current_term: 1 leader_uuid: "6954c779ad8b42ef9223c0dc56ea5f5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6954c779ad8b42ef9223c0dc56ea5f5b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 45647 } } }
I20260928 14:40:31.319293 3802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:31.319327 3807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:40:31.319947 3807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:40:31.321688 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:31.309315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50870 (local address 127.22.13.126:45647)
0928 14:40:31.309529 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:31.309541 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:31.309579 (+ 38us) server_negotiation.cc:407] Connection header received
0928 14:40:31.310150 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:31.310161 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:31.310249 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:31.310421 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:31.311070 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:31.312478 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:31.313328 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:31.313924 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:31.314075 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:31.316866 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:31.316870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:31.316870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:31.316885 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:31.319127 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:31.320057 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:31.320060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:31.320060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:31.320170 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:31.320428 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:31.320430 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:31.320430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:31.320775 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:31.320938 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:31.321247 (+ 309us) server_negotiation.cc:299] Negotiation successful
0928 14:40:31.321520 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":87,"threads_started":1}
I20260928 14:40:31.322211 3807 catalog_manager.cc:1347] Generated new cluster ID: 0da40296742a4104831c053cb3192f89
I20260928 14:40:31.322290 3807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:40:31.322811 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:40:31.336220 3807 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:40:31.337026 3807 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:40:31.343144 3807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6954c779ad8b42ef9223c0dc56ea5f5b: Generated new TSK 0
I20260928 14:40:31.343398 3807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:40:31.376845 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 3727
2026-09-28T14:40:31Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1347 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:40:31 dist-test-slave-wwwb krb5kdc[3822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:40:31 dist-test-slave-wwwb krb5kdc[3822](info): set up 2 sockets
Sep 28 14:40:31 dist-test-slave-wwwb krb5kdc[3822](info): commencing operation
krb5kdc: starting...
W20260928 14:40:32.414208 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:40:32 dist-test-slave-wwwb krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:40:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:40:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:40:32.532929 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41033
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:41033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:32.615319 3835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:32.615597 3835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:32.615638 3835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:32.616915 3835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:40:32.617014 3835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:32.617038 3835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:32.617057 3835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:40:32.617089 3835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:40:32.618741 3835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:41033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:32.619072 3835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:32.619315 3835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:32.624274 3843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:32.624439 3835 server_base.cc:1034] running on GCE node
W20260928 14:40:32.624295 3841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:32.624257 3840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:32.624799 3835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:32.625118 3835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:32.626276 3835 hybrid_clock.cc:648] HybridClock initialized: now 1790606432626261 us; error 45 us; skew 500 ppm
Sep 28 14:40:32 dist-test-slave-wwwb krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606432, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:32.632349 3835 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:40:32.632834 3835 webserver.cc:460] Webserver started at http://127.22.13.126:43083/ using document root <none> and password file <none>
I20260928 14:40:32.633101 3835 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:32.633174 3835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:32.633327 3835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:40:32.634528 3835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "8930beb078004e3c99c96a45245e53ee"
format_stamp: "Formatted at 2026-09-28 14:40:32 on dist-test-slave-wwwb"
server_key: "b6537596dea83803ddd1c4dbb6bce064"
server_key_iv: "f7e18751bf98d7ce03e7a47fc1c6f8e6"
server_key_version: "encryptionkey@0"
I20260928 14:40:32.634913 3835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "8930beb078004e3c99c96a45245e53ee"
format_stamp: "Formatted at 2026-09-28 14:40:32 on dist-test-slave-wwwb"
server_key: "b6537596dea83803ddd1c4dbb6bce064"
server_key_iv: "f7e18751bf98d7ce03e7a47fc1c6f8e6"
server_key_version: "encryptionkey@0"
I20260928 14:40:32.636298 3835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:40:32.637055 3850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:32.637238 3835 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:40:32.637333 3835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "8930beb078004e3c99c96a45245e53ee"
format_stamp: "Formatted at 2026-09-28 14:40:32 on dist-test-slave-wwwb"
server_key: "b6537596dea83803ddd1c4dbb6bce064"
server_key_iv: "f7e18751bf98d7ce03e7a47fc1c6f8e6"
server_key_version: "encryptionkey@0"
I20260928 14:40:32.637416 3835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:32.668236 3835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:32.670280 3835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:32.670457 3835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:32.674468 3835 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:41033
I20260928 14:40:32.674525 3902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:41033 every 8 connection(s)
I20260928 14:40:32.674921 3835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:40:32.675985 3903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:32.678481 3903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee: Bootstrap starting.
I20260928 14:40:32.678726 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 3835
I20260928 14:40:32.678850 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:40:32.679069 22581 external_mini_cluster.cc:1428] Setting key 9c795fbcf4821229f7fbeef19c96ca4e
I20260928 14:40:32.679299 3903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:32.679683 3903 log.cc:824] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:32.680482 3903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee: No bootstrap required, opened a new log
I20260928 14:40:32.682922 3903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } }
I20260928 14:40:32.683110 3903 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:32.683139 3903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8930beb078004e3c99c96a45245e53ee, State: Initialized, Role: FOLLOWER
I20260928 14:40:32.683302 3903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } }
I20260928 14:40:32.683391 3903 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:40:32.683415 3903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:40:32.683483 3903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:32.684221 3903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } }
I20260928 14:40:32.684330 3903 leader_election.cc:302] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8930beb078004e3c99c96a45245e53ee; no voters:
I20260928 14:40:32.684504 3903 leader_election.cc:288] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:40:32.684561 3908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:40:32.684811 3903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:40:32.684906 3908 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 1 LEADER]: Becoming Leader. State: Replica: 8930beb078004e3c99c96a45245e53ee, State: Running, Role: LEADER
I20260928 14:40:32.685022 3908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } }
I20260928 14:40:32.685415 3908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8930beb078004e3c99c96a45245e53ee. Latest consensus state: current_term: 1 leader_uuid: "8930beb078004e3c99c96a45245e53ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } } }
I20260928 14:40:32.685424 3910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8930beb078004e3c99c96a45245e53ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } } }
I20260928 14:40:32.685575 3910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:32.685815 3908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:32.686017 3917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:40:32.686918 3917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 28 14:40:32 dist-test-slave-wwwb krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:32.689577 3917 catalog_manager.cc:1347] Generated new cluster ID: a2039240fb5e40468033b8baab06fbef
I20260928 14:40:32.689633 3917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:40:32.692263 3906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:32.679653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44098 (local address 127.22.13.126:41033)
0928 14:40:32.679870 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:32.679880 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:32.679916 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:40:32.680526 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:32.680536 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:32.680603 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:32.680766 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:32.681358 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:32.682877 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:32.683850 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:32.684510 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:32.684672 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:32.687705 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:32.687709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:32.687710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:32.687722 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:32.690605 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:32.691011 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:32.691015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:32.691016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:32.691119 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:32.691347 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:32.691348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:32.691348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:32.691625 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:32.691834 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:32.691914 (+ 80us) server_negotiation.cc:299] Negotiation successful
0928 14:40:32.692065 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":88,"threads_started":1}
I20260928 14:40:32.705462 3917 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:40:32.706126 3917 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:40:32.717770 3917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee: Generated new TSK 0
I20260928 14:40:32.718060 3917 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:40:32.747285 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:41033
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:32.847203 3929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:32.847435 3929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:32.847460 3929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:32.848697 3929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:32.848747 3929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:32.848774 3929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:40:32.850293 3929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:41033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:32.850569 3929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:32.850823 3929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:32.855914 3934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:32.856006 3929 server_base.cc:1034] running on GCE node
W20260928 14:40:32.855914 3937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:32.855962 3935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:32.856343 3929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:32.856650 3929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:32.857822 3929 hybrid_clock.cc:648] HybridClock initialized: now 1790606432857814 us; error 36 us; skew 500 ppm
Sep 28 14:40:32 dist-test-slave-wwwb krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606432, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:32.860482 3929 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:40:32.860981 3929 webserver.cc:460] Webserver started at http://127.22.13.65:38769/ using document root <none> and password file <none>
I20260928 14:40:32.861220 3929 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:32.861299 3929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:32.861456 3929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:40:32.862627 3929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "0ec98f35ee1546dea30d1610662f24d7"
format_stamp: "Formatted at 2026-09-28 14:40:32 on dist-test-slave-wwwb"
server_key: "0b75f8158b42cca2d8d8c956d106a05f"
server_key_iv: "309da32ea8630b115f1c7c346b9716d5"
server_key_version: "encryptionkey@0"
I20260928 14:40:32.862995 3929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "0ec98f35ee1546dea30d1610662f24d7"
format_stamp: "Formatted at 2026-09-28 14:40:32 on dist-test-slave-wwwb"
server_key: "0b75f8158b42cca2d8d8c956d106a05f"
server_key_iv: "309da32ea8630b115f1c7c346b9716d5"
server_key_version: "encryptionkey@0"
I20260928 14:40:32.864377 3929 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:40:32.865127 3944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:32.865305 3929 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:40:32.865412 3929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "0ec98f35ee1546dea30d1610662f24d7"
format_stamp: "Formatted at 2026-09-28 14:40:32 on dist-test-slave-wwwb"
server_key: "0b75f8158b42cca2d8d8c956d106a05f"
server_key_iv: "309da32ea8630b115f1c7c346b9716d5"
server_key_version: "encryptionkey@0"
I20260928 14:40:32.865489 3929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:32.877364 3929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:32.880076 3929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:32.880275 3929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:32.880548 3929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:32.880891 3929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:40:32.880945 3929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:32.880982 3929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:40:32.881028 3929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:32.887604 3929 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:43987
I20260928 14:40:32.887646 4057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:43987 every 8 connection(s)
I20260928 14:40:32.887977 3929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:40:32.893603 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 3929
I20260928 14:40:32.893739 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:40:32.893972 22581 external_mini_cluster.cc:1428] Setting key 215fd23fa168e688f2f2e37cfb2c8a75
Sep 28 14:40:32 dist-test-slave-wwwb krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606432, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:32.897822 3906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:32.888672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:60813 (local address 127.22.13.126:41033)
0928 14:40:32.888717 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:32.888719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:32.888922 (+ 203us) server_negotiation.cc:407] Connection header received
0928 14:40:32.889443 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:32.889444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:32.889453 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:32.889516 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:32.890584 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:32.891085 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:32.892092 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:32.892312 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:32.892366 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:32.894810 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:32.894812 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:32.894812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:32.894819 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:32.896288 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:32.896755 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:32.896757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:32.896757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:32.896826 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:32.897102 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:32.897103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:32.897103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:32.897234 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:32.897312 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:32.897606 (+ 294us) server_negotiation.cc:299] Negotiation successful
0928 14:40:32.897708 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:40:32.898154 4060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:32.888628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:60813)
0928 14:40:32.888791 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:32.888836 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:32.889357 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:32.889576 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:32.889580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:32.890272 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:32.890488 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:32.890494 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:32.891178 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:32.891179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:32.892011 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:32.892014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:32.892391 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:32.892392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:32.892502 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:32.893046 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:32.893055 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:32.894676 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:32.896400 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:32.896408 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:32.896410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:32.896655 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:32.896905 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:32.896905 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:32.896908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:32.897026 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:32.897302 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:32.897310 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:32.897458 (+ 148us) client_negotiation.cc:769] Sending connection context
0928 14:40:32.897587 (+ 129us) client_negotiation.cc:240] Negotiation successful
0928 14:40:32.897787 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260928 14:40:32.898787 4058 heartbeater.cc:347] Connected to a master server at 127.22.13.126:41033
I20260928 14:40:32.898910 4058 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:32.899163 4058 heartbeater.cc:511] Master 127.22.13.126:41033 requested a full tablet report, sending...
I20260928 14:40:32.899755 3867 ts_manager.cc:194] Registered new tserver with Master: 0ec98f35ee1546dea30d1610662f24d7 (127.22.13.65:43987)
I20260928 14:40:32.900393 3867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:60813
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:40:32.920686 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:41033
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:33.006812 4063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:33.007035 4063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:33.007061 4063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:33.008654 4063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:33.008705 4063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:33.008764 4063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:40:33.010953 4063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:41033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:33.011262 4063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:33.011514 4063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:33.016958 4069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:33.017027 4071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:33.016958 4068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:33.017386 4063 server_base.cc:1034] running on GCE node
I20260928 14:40:33.017620 4063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:33.017889 4063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:33.019084 4063 hybrid_clock.cc:648] HybridClock initialized: now 1790606433019063 us; error 36 us; skew 500 ppm
Sep 28 14:40:33 dist-test-slave-wwwb krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606433, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:33.021571 4063 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:40:33.022114 4063 webserver.cc:460] Webserver started at http://127.22.13.66:33537/ using document root <none> and password file <none>
I20260928 14:40:33.022354 4063 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:33.022420 4063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:33.022574 4063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:40:33.023658 4063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "f285265e654e42bfa1bc49224de0ac2d"
format_stamp: "Formatted at 2026-09-28 14:40:33 on dist-test-slave-wwwb"
server_key: "55d221abe0a29a24f826ef3287c025df"
server_key_iv: "bb3cbf1d60ac0f6bf1eeb7371117cf36"
server_key_version: "encryptionkey@0"
I20260928 14:40:33.024039 4063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "f285265e654e42bfa1bc49224de0ac2d"
format_stamp: "Formatted at 2026-09-28 14:40:33 on dist-test-slave-wwwb"
server_key: "55d221abe0a29a24f826ef3287c025df"
server_key_iv: "bb3cbf1d60ac0f6bf1eeb7371117cf36"
server_key_version: "encryptionkey@0"
I20260928 14:40:33.025434 4063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:40:33.026306 4078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:33.026481 4063 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:33.026571 4063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "f285265e654e42bfa1bc49224de0ac2d"
format_stamp: "Formatted at 2026-09-28 14:40:33 on dist-test-slave-wwwb"
server_key: "55d221abe0a29a24f826ef3287c025df"
server_key_iv: "bb3cbf1d60ac0f6bf1eeb7371117cf36"
server_key_version: "encryptionkey@0"
I20260928 14:40:33.026628 4063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:33.036846 4063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:33.039211 4063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:33.039328 4063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:33.039541 4063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:33.039893 4063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:40:33.039927 4063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:33.039950 4063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:40:33.039996 4063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:33.046053 4063 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:33767
I20260928 14:40:33.046120 4191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:33767 every 8 connection(s)
I20260928 14:40:33.046451 4063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
Sep 28 14:40:33 dist-test-slave-wwwb krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606433, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:33.056053 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 4063
I20260928 14:40:33.056178 3906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.047217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:36425 (local address 127.22.13.126:41033)
0928 14:40:33.047260 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.047262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.047529 (+ 267us) server_negotiation.cc:407] Connection header received
0928 14:40:33.048082 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.048083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.048095 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.048154 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:33.049174 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.049679 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.050817 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.051035 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.051112 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.053533 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:33.053535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:33.053535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:33.053543 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:33.054861 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:33.055263 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:33.055264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:33.055264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:33.055314 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:33.055531 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:33.055532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:33.055532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:33.055640 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:33.055695 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.055981 (+ 286us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.056083 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:40:33.056200 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:40:33.056385 4194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.047213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:36425)
0928 14:40:33.047400 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.047437 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:33.047971 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:33.048235 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:33.048240 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:33.048848 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:33.049057 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.049062 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.049783 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.049784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.050710 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.050712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.051122 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.051123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.051265 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.051880 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:33.051889 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:33.053431 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:33.054943 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:33.054951 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:33.054953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:33.055194 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:33.055372 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:33.055372 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:33.055373 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:33.055478 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:33.055702 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:33.055706 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:33.055832 (+ 126us) client_negotiation.cc:769] Sending connection context
0928 14:40:33.055953 (+ 121us) client_negotiation.cc:240] Negotiation successful
0928 14:40:33.056124 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
I20260928 14:40:33.056401 22581 external_mini_cluster.cc:1428] Setting key 7ff80b81ca88b00ed20cc518adea0ff5
I20260928 14:40:33.056931 4192 heartbeater.cc:347] Connected to a master server at 127.22.13.126:41033
I20260928 14:40:33.057047 4192 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:33.057276 4192 heartbeater.cc:511] Master 127.22.13.126:41033 requested a full tablet report, sending...
I20260928 14:40:33.057765 3866 ts_manager.cc:194] Registered new tserver with Master: f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767)
I20260928 14:40:33.058229 3866 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:36425
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:40:33.080966 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:41033
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:33.165072 4197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:33.165357 4197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:33.165396 4197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:33.166672 4197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:33.166752 4197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:33.166795 4197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:40:33.168329 4197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:41033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:33.168625 4197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:33.168926 4197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:33.174059 4205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:33.174115 4202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:33.174059 4203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:33.174157 4197 server_base.cc:1034] running on GCE node
I20260928 14:40:33.174434 4197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:33.174647 4197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:33.175818 4197 hybrid_clock.cc:648] HybridClock initialized: now 1790606433175800 us; error 38 us; skew 500 ppm
Sep 28 14:40:33 dist-test-slave-wwwb krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606433, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:33.178359 4197 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:40:33.178813 4197 webserver.cc:460] Webserver started at http://127.22.13.67:36433/ using document root <none> and password file <none>
I20260928 14:40:33.179081 4197 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:33.179154 4197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:33.179306 4197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:40:33.180405 4197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "22a55c393f224764b560f65f0526d943"
format_stamp: "Formatted at 2026-09-28 14:40:33 on dist-test-slave-wwwb"
server_key: "51524c1ecedb54c66fa7e7c3bc13115a"
server_key_iv: "39d55f54d1fe2458be655d4a088a4f11"
server_key_version: "encryptionkey@0"
I20260928 14:40:33.180776 4197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "22a55c393f224764b560f65f0526d943"
format_stamp: "Formatted at 2026-09-28 14:40:33 on dist-test-slave-wwwb"
server_key: "51524c1ecedb54c66fa7e7c3bc13115a"
server_key_iv: "39d55f54d1fe2458be655d4a088a4f11"
server_key_version: "encryptionkey@0"
I20260928 14:40:33.182201 4197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 14:40:33.182933 4212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:33.183131 4197 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:40:33.183236 4197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "22a55c393f224764b560f65f0526d943"
format_stamp: "Formatted at 2026-09-28 14:40:33 on dist-test-slave-wwwb"
server_key: "51524c1ecedb54c66fa7e7c3bc13115a"
server_key_iv: "39d55f54d1fe2458be655d4a088a4f11"
server_key_version: "encryptionkey@0"
I20260928 14:40:33.183315 4197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:33.193812 4197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:33.195971 4197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:33.196135 4197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:33.196378 4197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:33.196695 4197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:40:33.196751 4197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:33.196794 4197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:40:33.196833 4197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:33.202917 4197 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:38323
I20260928 14:40:33.202962 4325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:38323 every 8 connection(s)
I20260928 14:40:33.203306 4197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:40:33.206148 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 4197
I20260928 14:40:33.206250 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:40:33.206394 22581 external_mini_cluster.cc:1428] Setting key 7b786634e4f17eec458dcde996393b70
Sep 28 14:40:33 dist-test-slave-wwwb krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606433, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:33.212792 3906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.204072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:47607 (local address 127.22.13.126:41033)
0928 14:40:33.204169 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.204171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.204361 (+ 190us) server_negotiation.cc:407] Connection header received
0928 14:40:33.204854 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.204855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.204863 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.204931 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:33.205895 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.206546 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.207908 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.208102 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.208161 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.210465 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:33.210466 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:33.210466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:33.210473 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:33.211661 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:33.212021 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:33.212021 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:33.212022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:33.212075 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:33.212277 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:33.212277 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:33.212277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:33.212378 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:33.212424 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.212664 (+ 240us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.212725 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 14:40:33.213035 4328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.204031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:47607)
0928 14:40:33.204217 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.204273 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:33.204772 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:33.204990 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:33.204993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:33.205558 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:33.205806 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.205811 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.206668 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.206670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.207827 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.207830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.208181 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.208182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.208305 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.208875 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:33.208884 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:33.210379 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:33.211735 (+ 1356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:33.211741 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:33.211742 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:33.211957 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:33.212130 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:33.212131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:33.212131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:33.212226 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:33.212429 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:33.212432 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:33.212550 (+ 118us) client_negotiation.cc:769] Sending connection context
0928 14:40:33.212640 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 14:40:33.212760 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":107,"threads_started":1}
I20260928 14:40:33.213676 4326 heartbeater.cc:347] Connected to a master server at 127.22.13.126:41033
I20260928 14:40:33.213794 4326 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:33.214027 4326 heartbeater.cc:511] Master 127.22.13.126:41033 requested a full tablet report, sending...
I20260928 14:40:33.214464 3866 ts_manager.cc:194] Registered new tserver with Master: 22a55c393f224764b560f65f0526d943 (127.22.13.67:38323)
I20260928 14:40:33.214857 3866 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:47607
I20260928 14:40:33.218343 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:40:33.225600 3906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.219586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44102 (local address 127.22.13.126:41033)
0928 14:40:33.219667 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.219669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.219769 (+ 100us) server_negotiation.cc:407] Connection header received
0928 14:40:33.219845 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.219847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.219856 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.219910 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:33.220477 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.220956 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.221693 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.221890 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.222012 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.223251 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:33.223252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:33.223252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:33.223260 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:33.224377 (+ 1117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:33.224739 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:33.224740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:33.224741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:33.224804 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:33.225015 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:33.225016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:33.225016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:33.225191 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:33.225303 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.225402 (+ 99us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.225486 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 14:40:33.227174 3866 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:40:33.228752 3866 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:40:33.235391 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.231249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:43987 (local address 127.0.0.1:51832)
0928 14:40:33.231620 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.231632 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:33.231678 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:33.232111 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:33.232112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:33.232114 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:33.232153 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.232154 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.233409 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.233410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.234279 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.234281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.235043 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.235044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.235134 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.235151 (+ 17us) client_negotiation.cc:769] Sending connection context
0928 14:40:33.235229 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:40:33.235281 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":37,"thread_start_us":97,"threads_started":1}
I20260928 14:40:33.235400 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.231696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51832 (local address 127.22.13.65:43987)
0928 14:40:33.231885 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.231889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.231898 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:40:33.231933 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.231935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.231950 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.232024 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:33.232232 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.233299 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.234389 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.234951 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.235025 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.235087 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.235253 (+ 166us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.235299 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":76,"threads_started":1}
I20260928 14:40:33.235922 4338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.231249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:38323 (local address 127.0.0.1:50674)
0928 14:40:33.231749 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.231757 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:33.231798 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:33.232072 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:33.232073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:33.232075 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:33.232126 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.232127 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.232994 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.232995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.234291 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.234294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.235337 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.235338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.235415 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.235420 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:40:33.235798 (+ 378us) client_negotiation.cc:240] Negotiation successful
0928 14:40:33.235848 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":2,"thread_start_us":97,"threads_started":1}
I20260928 14:40:33.236260 4341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.231488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50674 (local address 127.22.13.67:38323)
0928 14:40:33.231694 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.231699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.231851 (+ 152us) server_negotiation.cc:407] Connection header received
0928 14:40:33.231896 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.231898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.231913 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.231984 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:33.232199 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.232878 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.234400 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.235166 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.235238 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.235291 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.235821 (+ 530us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.235864 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":107,"threads_started":1}
I20260928 14:40:33.236297 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.231229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:33767 (local address 127.0.0.1:34908)
0928 14:40:33.231332 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.231374 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:33.231442 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:33.231842 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:33.231846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:33.231849 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:33.231933 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.231934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.233380 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.233381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.234799 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.234802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.236067 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.236068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.236148 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.236152 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:40:33.236183 (+ 31us) client_negotiation.cc:240] Negotiation successful
0928 14:40:33.236226 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":24,"thread_start_us":61,"threads_started":1}
I20260928 14:40:33.236234 3992 tablet_service.cc:1467] Processing CreateTablet for tablet abf27f74f1cb490a969142de9ac749dc (DEFAULT_TABLE table=test-table [id=7f14648de444424c98129ed5b55870a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:40:33.236652 3992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abf27f74f1cb490a969142de9ac749dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:33.236907 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.231409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34908 (local address 127.22.13.66:33767)
0928 14:40:33.231621 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.231626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.231637 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:40:33.231682 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.231685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.231700 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.232012 (+ 312us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:33.232090 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.233269 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.234892 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.235968 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.236673 (+ 705us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.236742 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.236782 (+ 40us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.236825 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":98,"threads_started":1}
I20260928 14:40:33.237058 4260 tablet_service.cc:1467] Processing CreateTablet for tablet abf27f74f1cb490a969142de9ac749dc (DEFAULT_TABLE table=test-table [id=7f14648de444424c98129ed5b55870a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:40:33.237380 4260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abf27f74f1cb490a969142de9ac749dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:33.237514 4126 tablet_service.cc:1467] Processing CreateTablet for tablet abf27f74f1cb490a969142de9ac749dc (DEFAULT_TABLE table=test-table [id=7f14648de444424c98129ed5b55870a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:40:33.237826 4126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abf27f74f1cb490a969142de9ac749dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:33.239506 4343 tablet_bootstrap.cc:492] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: Bootstrap starting.
I20260928 14:40:33.240200 4343 tablet_bootstrap.cc:654] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:33.240762 4344 tablet_bootstrap.cc:492] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943: Bootstrap starting.
I20260928 14:40:33.240834 4343 log.cc:824] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:33.241250 4345 tablet_bootstrap.cc:492] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d: Bootstrap starting.
I20260928 14:40:33.241557 4344 tablet_bootstrap.cc:654] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:33.241838 4345 tablet_bootstrap.cc:654] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:33.242120 4344 log.cc:824] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:33.242484 4345 log.cc:824] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:33.242620 4343 tablet_bootstrap.cc:492] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: No bootstrap required, opened a new log
I20260928 14:40:33.242697 4343 ts_tablet_manager.cc:1397] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:40:33.242936 4344 tablet_bootstrap.cc:492] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943: No bootstrap required, opened a new log
I20260928 14:40:33.243005 4344 ts_tablet_manager.cc:1397] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:40:33.243222 4345 tablet_bootstrap.cc:492] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d: No bootstrap required, opened a new log
I20260928 14:40:33.243309 4345 ts_tablet_manager.cc:1397] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260928 14:40:33.244760 4343 raft_consensus.cc:348] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.244796 4345 raft_consensus.cc:348] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.244939 4343 raft_consensus.cc:374] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:33.244964 4345 raft_consensus.cc:374] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:33.244980 4343 raft_consensus.cc:729] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec98f35ee1546dea30d1610662f24d7, State: Initialized, Role: FOLLOWER
I20260928 14:40:33.245002 4345 raft_consensus.cc:729] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f285265e654e42bfa1bc49224de0ac2d, State: Initialized, Role: FOLLOWER
I20260928 14:40:33.244907 4344 raft_consensus.cc:348] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.245033 4344 raft_consensus.cc:374] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:33.245064 4344 raft_consensus.cc:729] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22a55c393f224764b560f65f0526d943, State: Initialized, Role: FOLLOWER
I20260928 14:40:33.245106 4343 consensus_queue.cc:260] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.245115 4345 consensus_queue.cc:260] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.245175 4344 consensus_queue.cc:260] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.245394 4343 ts_tablet_manager.cc:1428] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:40:33.245409 4344 ts_tablet_manager.cc:1428] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:40:33.245394 4345 ts_tablet_manager.cc:1428] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 14:40:33.245404 4192 heartbeater.cc:503] Master 127.22.13.126:41033 was elected leader, sending a full tablet report...
I20260928 14:40:33.245509 4326 heartbeater.cc:503] Master 127.22.13.126:41033 was elected leader, sending a full tablet report...
I20260928 14:40:33.245550 4058 heartbeater.cc:503] Master 127.22.13.126:41033 was elected leader, sending a full tablet report...
I20260928 14:40:33.254993 4349 raft_consensus.cc:482] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:40:33.255097 4349 raft_consensus.cc:504] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.255517 4349 leader_election.cc:288] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767), 22a55c393f224764b560f65f0526d943 (127.22.13.67:38323)
I20260928 14:40:33.258415 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.255813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:53095 (local address 127.22.13.66:33767)
0928 14:40:33.255861 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.255863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.255873 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:40:33.255903 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.255904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.255916 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.255975 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:33.256166 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.256681 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.257604 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.258045 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.258102 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.258109 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.258293 (+ 184us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.258342 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:40:33.258445 4341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.255786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:54907 (local address 127.22.13.67:38323)
0928 14:40:33.255827 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.255829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.255989 (+ 160us) server_negotiation.cc:407] Connection header received
0928 14:40:33.256014 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.256015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.256024 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.256076 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:33.256258 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.256707 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.257645 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.258062 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.258171 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.258179 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.258342 (+ 163us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.258383 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:40:33.258430 4060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.255691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:33767 (local address 127.22.13.65:53095)
0928 14:40:33.255763 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.255777 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:33.255832 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:33.255987 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:33.255988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:33.255990 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:33.256086 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.256088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.256759 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.256760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.257513 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.257515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.258119 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.258120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.258219 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.258223 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:40:33.258279 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 14:40:33.258336 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260928 14:40:33.258719 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.255691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:38323 (local address 127.22.13.65:54907)
0928 14:40:33.255878 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.255889 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:33.255952 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:33.256096 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:33.256097 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:33.256098 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:33.256189 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.256190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.256814 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.256815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.257524 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.257526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.258177 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.258178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.258247 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.258250 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:40:33.258327 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:40:33.258365 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":18,"thread_start_us":108,"threads_started":1}
I20260928 14:40:33.258807 4146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abf27f74f1cb490a969142de9ac749dc" candidate_uuid: "0ec98f35ee1546dea30d1610662f24d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f285265e654e42bfa1bc49224de0ac2d" is_pre_election: true
I20260928 14:40:33.259002 4146 raft_consensus.cc:2454] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec98f35ee1546dea30d1610662f24d7 in term 0.
I20260928 14:40:33.259042 4280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abf27f74f1cb490a969142de9ac749dc" candidate_uuid: "0ec98f35ee1546dea30d1610662f24d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22a55c393f224764b560f65f0526d943" is_pre_election: true
I20260928 14:40:33.259207 4280 raft_consensus.cc:2454] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec98f35ee1546dea30d1610662f24d7 in term 0.
I20260928 14:40:33.259236 3946 leader_election.cc:302] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec98f35ee1546dea30d1610662f24d7, f285265e654e42bfa1bc49224de0ac2d; no voters:
I20260928 14:40:33.259344 4349 raft_consensus.cc:2781] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:40:33.259428 4349 raft_consensus.cc:482] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:40:33.259462 4349 raft_consensus.cc:3037] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:33.260162 4349 raft_consensus.cc:504] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.260311 4349 leader_election.cc:288] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 1 election: Requested vote from peers f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767), 22a55c393f224764b560f65f0526d943 (127.22.13.67:38323)
I20260928 14:40:33.260440 4280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abf27f74f1cb490a969142de9ac749dc" candidate_uuid: "0ec98f35ee1546dea30d1610662f24d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22a55c393f224764b560f65f0526d943"
I20260928 14:40:33.260466 4146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abf27f74f1cb490a969142de9ac749dc" candidate_uuid: "0ec98f35ee1546dea30d1610662f24d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f285265e654e42bfa1bc49224de0ac2d"
I20260928 14:40:33.260535 4280 raft_consensus.cc:3037] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:33.260560 4146 raft_consensus.cc:3037] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:33.261349 4146 raft_consensus.cc:2454] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec98f35ee1546dea30d1610662f24d7 in term 1.
I20260928 14:40:33.261379 4280 raft_consensus.cc:2454] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec98f35ee1546dea30d1610662f24d7 in term 1.
I20260928 14:40:33.261526 3946 leader_election.cc:302] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec98f35ee1546dea30d1610662f24d7, f285265e654e42bfa1bc49224de0ac2d; no voters:
I20260928 14:40:33.261610 4349 raft_consensus.cc:2781] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:40:33.261777 4349 raft_consensus.cc:686] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ec98f35ee1546dea30d1610662f24d7, State: Running, Role: LEADER
I20260928 14:40:33.261897 4349 consensus_queue.cc:237] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.262784 3865 catalog_manager.cc:5495] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec98f35ee1546dea30d1610662f24d7 (127.22.13.65). New cstate: current_term: 1 leader_uuid: "0ec98f35ee1546dea30d1610662f24d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } health_report { overall_health: HEALTHY } } }
I20260928 14:40:33.273173 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.270682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51834 (local address 127.22.13.65:43987)
0928 14:40:33.270735 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.270737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.270747 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:40:33.270809 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.270810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.270832 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.270916 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:40:33.271077 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.271544 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.272319 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.272529 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.272619 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.272805 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:40:33.272917 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:40:33.273015 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.273032 (+ 17us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.273062 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 14:40:33.275136 4146 raft_consensus.cc:1261] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d [term 1 FOLLOWER]: Refusing update from remote peer 0ec98f35ee1546dea30d1610662f24d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:40:33.275178 4280 raft_consensus.cc:1261] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943 [term 1 FOLLOWER]: Refusing update from remote peer 0ec98f35ee1546dea30d1610662f24d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:40:33.275538 4349 consensus_queue.cc:1035] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:40:33.275724 4353 consensus_queue.cc:1035] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:40:33.278306 4356 mvcc.cc:205] Tried to move back new op lower bound from 7334323950692188160 to 7334323950641700864. Current Snapshot: MvccSnapshot[applied={T|T < 7334323950692188160}]
I20260928 14:40:33.278743 4357 mvcc.cc:205] Tried to move back new op lower bound from 7334323950692188160 to 7334323950641700864. Current Snapshot: MvccSnapshot[applied={T|T < 7334323950692188160}]
I20260928 14:40:33.280658 3865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:40:33.280764 3865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:40:33.281581 3865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee: Sending DeleteTablet for 3 replicas of tablet abf27f74f1cb490a969142de9ac749dc
I20260928 14:40:33.281921 3992 tablet_service.cc:1514] Processing DeleteTablet for tablet abf27f74f1cb490a969142de9ac749dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:40:33 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:51832
I20260928 14:40:33.282004 4260 tablet_service.cc:1514] Processing DeleteTablet for tablet abf27f74f1cb490a969142de9ac749dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:40:33 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:50674
I20260928 14:40:33.282181 4366 tablet_replica.cc:330] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: stopping tablet replica
I20260928 14:40:33.282197 4126 tablet_service.cc:1514] Processing DeleteTablet for tablet abf27f74f1cb490a969142de9ac749dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:40:33 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:34908
I20260928 14:40:33.282292 4366 raft_consensus.cc:2229] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:40:33.282425 4366 raft_consensus.cc:2258] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:40:33.282543 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 3929
I20260928 14:40:33.282826 4366 ts_tablet_manager.cc:1905] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:40:33.286101 4368 tablet_replica.cc:330] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d: stopping tablet replica
I20260928 14:40:33.286242 4368 raft_consensus.cc:2229] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:40:33.286365 4368 raft_consensus.cc:2258] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:40:33.286562 4367 tablet_replica.cc:330] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943: stopping tablet replica
I20260928 14:40:33.286695 4367 raft_consensus.cc:2229] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:40:33.286806 4367 raft_consensus.cc:2258] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:40:33.286909 4368 ts_tablet_manager.cc:1905] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:40:33.287532 4367 ts_tablet_manager.cc:1905] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:40:33.288600 4368 ts_tablet_manager.cc:1918] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:40:33.288679 4368 log.cc:1196] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/abf27f74f1cb490a969142de9ac749dc
I20260928 14:40:33.288944 4368 ts_tablet_manager.cc:1939] T abf27f74f1cb490a969142de9ac749dc P f285265e654e42bfa1bc49224de0ac2d: Deleting consensus metadata
I20260928 14:40:33.289414 3852 catalog_manager.cc:4841] TS f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767): tablet abf27f74f1cb490a969142de9ac749dc (table test-table [id=7f14648de444424c98129ed5b55870a1]) successfully deleted
I20260928 14:40:33.289897 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 4063
I20260928 14:40:33.289888 4367 ts_tablet_manager.cc:1918] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260928 14:40:33.294142 3853 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:43987 (error 108)
I20260928 14:40:33.294193 4367 log.cc:1196] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/abf27f74f1cb490a969142de9ac749dc
I20260928 14:40:33.294548 4367 ts_tablet_manager.cc:1939] T abf27f74f1cb490a969142de9ac749dc P 22a55c393f224764b560f65f0526d943: Deleting consensus metadata
I20260928 14:40:33.294705 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.294575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:43987 (local address 127.0.0.1:51844)
0928 14:40:33.294629 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.294661 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 14:40:33.294845 3853 catalog_manager.cc:4584] TS 0ec98f35ee1546dea30d1610662f24d7 (127.22.13.65:43987): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet abf27f74f1cb490a969142de9ac749dc: Network error: Client connection negotiation failed: client connection to 127.22.13.65:43987: connect: Connection refused (error 111)
I20260928 14:40:33.295053 3851 catalog_manager.cc:4841] TS 22a55c393f224764b560f65f0526d943 (127.22.13.67:38323): tablet abf27f74f1cb490a969142de9ac749dc (table test-table [id=7f14648de444424c98129ed5b55870a1]) successfully deleted
I20260928 14:40:33.295907 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 4197
I20260928 14:40:33.301033 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 3835
I20260928 14:40:33.306388 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41033
--webserver_interface=127.22.13.126
--webserver_port=43083
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:41033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:33.389365 4370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:33.389600 4370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:33.389623 4370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:33.390885 4370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:40:33.390931 4370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:33.390945 4370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:33.390959 4370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:40:33.390970 4370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:40:33.392539 4370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:41033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=43083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:33.392753 4370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:33.392935 4370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:33.402206 4379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:33.402213 4377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:33.402231 4376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:33.402518 4370 server_base.cc:1034] running on GCE node
I20260928 14:40:33.402808 4370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:33.403213 4370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:33.404387 4370 hybrid_clock.cc:648] HybridClock initialized: now 1790606433404369 us; error 39 us; skew 500 ppm
Sep 28 14:40:33 dist-test-slave-wwwb krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606433, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:33.407244 4370 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:40:33.407704 4370 webserver.cc:460] Webserver started at http://127.22.13.126:43083/ using document root <none> and password file <none>
I20260928 14:40:33.407913 4370 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:33.407968 4370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:33.409403 4370 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:33.410959 4386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:33.411144 4370 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:33.411242 4370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "8930beb078004e3c99c96a45245e53ee"
format_stamp: "Formatted at 2026-09-28 14:40:32 on dist-test-slave-wwwb"
server_key: "b6537596dea83803ddd1c4dbb6bce064"
server_key_iv: "f7e18751bf98d7ce03e7a47fc1c6f8e6"
server_key_version: "encryptionkey@0"
I20260928 14:40:33.411747 4370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:33.435632 4370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:33.437625 4370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:33.437825 4370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:33.442106 4370 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:41033
I20260928 14:40:33.442147 4438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:41033 every 8 connection(s)
I20260928 14:40:33.442581 4370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:40:33.444361 4439 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:40:33.445094 4439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee: Bootstrap starting.
I20260928 14:40:33.451759 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 4370
I20260928 14:40:33.452100 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:43987
--local_ip_for_outbound_sockets=127.22.13.65
--tserver_master_addrs=127.22.13.126:41033
--webserver_port=38769
--webserver_interface=127.22.13.65
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:40:33.454197 4439 log.cc:824] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:33.456403 4439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:40:33.456718 4439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee: Bootstrap complete.
I20260928 14:40:33.459208 4439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } }
I20260928 14:40:33.459494 4439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8930beb078004e3c99c96a45245e53ee, State: Initialized, Role: FOLLOWER
I20260928 14:40:33.459705 4439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } }
I20260928 14:40:33.459805 4439 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:40:33.459864 4439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:40:33.459913 4439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:40:33.461030 4439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } }
I20260928 14:40:33.461159 4439 leader_election.cc:302] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8930beb078004e3c99c96a45245e53ee; no voters:
I20260928 14:40:33.461393 4439 leader_election.cc:288] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 14:40:33.461431 4443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 2 FOLLOWER]: Leader election won for term 2
I20260928 14:40:33.461542 4443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 2 LEADER]: Becoming Leader. State: Replica: 8930beb078004e3c99c96a45245e53ee, State: Running, Role: LEADER
I20260928 14:40:33.461678 4439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:40:33.461707 4443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } }
I20260928 14:40:33.462229 4445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8930beb078004e3c99c96a45245e53ee. Latest consensus state: current_term: 2 leader_uuid: "8930beb078004e3c99c96a45245e53ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } } }
I20260928 14:40:33.462239 4444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8930beb078004e3c99c96a45245e53ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } } }
I20260928 14:40:33.462355 4445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:33.462366 4444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:33.462826 4451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:40:33.463805 4451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:40:33.463954 4451 catalog_manager.cc:1259] Loaded cluster ID: a2039240fb5e40468033b8baab06fbef
I20260928 14:40:33.464196 4451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:40:33.464488 4451 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260928 14:40:33.469208 4451 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260928 14:40:33.472175 4451 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:40:33.472441 4451 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee: Loaded TSK: 0
I20260928 14:40:33.472707 4451 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 14:40:33.542490 4441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:33.542732 4441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:33.542773 4441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:33.544051 4441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:33.544137 4441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:33.544188 4441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:40:33.545811 4441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:43987
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=38769
--tserver_master_addrs=127.22.13.126:41033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:33.546141 4441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:33.546398 4441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:33.551537 4466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:33.551543 4467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:33.551542 4469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:33.552608 4441 server_base.cc:1034] running on GCE node
I20260928 14:40:33.552830 4441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:33.553103 4441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:33.554280 4441 hybrid_clock.cc:648] HybridClock initialized: now 1790606433554261 us; error 34 us; skew 500 ppm
Sep 28 14:40:33 dist-test-slave-wwwb krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606433, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:33.557032 4441 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:40:33.557555 4441 webserver.cc:460] Webserver started at http://127.22.13.65:38769/ using document root <none> and password file <none>
I20260928 14:40:33.557825 4441 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:33.557912 4441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:33.559461 4441 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:33.560304 4476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:33.560485 4441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:33.560593 4441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "0ec98f35ee1546dea30d1610662f24d7"
format_stamp: "Formatted at 2026-09-28 14:40:32 on dist-test-slave-wwwb"
server_key: "0b75f8158b42cca2d8d8c956d106a05f"
server_key_iv: "309da32ea8630b115f1c7c346b9716d5"
server_key_version: "encryptionkey@0"
I20260928 14:40:33.560974 4441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:33.582739 4441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:33.585096 4441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:33.585282 4441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:33.585531 4441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:33.586081 4483 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 14:40:33.588006 4441 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 14:40:33.588070 4441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 14:40:33.588119 4441 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 14:40:33.588769 4441 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 14:40:33.588827 4441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260928 14:40:33.588900 4483 tablet_bootstrap.cc:492] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: Bootstrap starting.
I20260928 14:40:33.590873 4483 log.cc:824] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:33.593910 4483 tablet_bootstrap.cc:492] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:40:33.594274 4483 tablet_bootstrap.cc:492] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: Bootstrap complete.
I20260928 14:40:33.594421 4483 ts_tablet_manager.cc:1397] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20260928 14:40:33.596251 4441 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:43987
I20260928 14:40:33.596629 4441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:40:33.596267 4483 raft_consensus.cc:348] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.596709 4483 raft_consensus.cc:729] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec98f35ee1546dea30d1610662f24d7, State: Initialized, Role: FOLLOWER
I20260928 14:40:33.596946 4483 consensus_queue.cc:260] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.597325 4483 ts_tablet_manager.cc:1428] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 14:40:33.597676 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 4441
I20260928 14:40:33.598003 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:33767
--local_ip_for_outbound_sockets=127.22.13.66
--tserver_master_addrs=127.22.13.126:41033
--webserver_port=33537
--webserver_interface=127.22.13.66
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:40:33.598176 4591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:43987 every 8 connection(s)
Sep 28 14:40:33 dist-test-slave-wwwb krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606433, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:33.614593 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.598875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:43291)
0928 14:40:33.599062 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.599109 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:33.599273 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:33.602567 (+ 3294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:33.602570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:33.602858 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:33.603063 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.603068 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.604960 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.604961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.606313 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.606317 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.607335 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.607337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.607480 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.608596 (+ 1116us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:33.608607 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:33.610621 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:33.612789 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:33.612797 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:33.612799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:33.613035 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:33.613320 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:33.613320 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:33.613320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:33.613422 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:33.613987 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:33.613995 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:33.614136 (+ 141us) client_negotiation.cc:769] Sending connection context
0928 14:40:33.614226 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 14:40:33.614417 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260928 14:40:33.614732 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.599117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:43291 (local address 127.22.13.126:41033)
0928 14:40:33.602094 (+ 2977us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.602105 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.602161 (+ 56us) server_negotiation.cc:407] Connection header received
0928 14:40:33.602294 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.602301 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.602353 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.602501 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:33.603224 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.604797 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.606421 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.607208 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.607346 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.610759 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:33.610764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:33.610764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:33.610778 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:33.612689 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:33.613122 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:33.613126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:33.613126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:33.613250 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:33.613559 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:33.613559 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:33.613560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:33.613870 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:33.614013 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.614328 (+ 315us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.614481 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2920,"thread_start_us":80,"threads_started":1}
I20260928 14:40:33.615636 4592 heartbeater.cc:347] Connected to a master server at 127.22.13.126:41033
I20260928 14:40:33.615779 4592 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:33.615978 4592 heartbeater.cc:511] Master 127.22.13.126:41033 requested a full tablet report, sending...
I20260928 14:40:33.616773 4403 ts_manager.cc:194] Registered new tserver with Master: 0ec98f35ee1546dea30d1610662f24d7 (127.22.13.65:43987)
I20260928 14:40:33.618180 4403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:43291
Sep 28 14:40:33 dist-test-slave-wwwb krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606433, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for kudu/127.22.13.65@KRBTEST.COM
I20260928 14:40:33.626161 4598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.617438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:43987 (local address 127.0.0.1:51856)
0928 14:40:33.617641 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.617677 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:33.617749 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:33.618102 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:33.618108 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:33.618496 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:33.618552 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.618555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.619632 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.619635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.620313 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.620316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.620837 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.620838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.620923 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.621448 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:33.621453 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:33.622975 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:33.624798 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:33.624807 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:33.624809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:33.625058 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:33.625322 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:33.625323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:33.625323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:33.625389 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:33.625701 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:33.625709 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:33.625765 (+ 56us) client_negotiation.cc:769] Sending connection context
0928 14:40:33.625871 (+ 106us) client_negotiation.cc:240] Negotiation successful
0928 14:40:33.626034 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":129,"threads_started":1}
I20260928 14:40:33.626161 4599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.617710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51856 (local address 127.22.13.65:43987)
0928 14:40:33.617904 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.617908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.617916 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:40:33.617993 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.617995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.618009 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.618099 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:33.618660 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.619536 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.620401 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.620750 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.620838 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.623079 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:33.623088 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:33.623088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:33.623104 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:33.624699 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:33.625175 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:33.625180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:33.625180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:33.625241 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:33.625458 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:33.625459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:33.625459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:33.625619 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:33.625731 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.625908 (+ 177us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.626034 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":89,"threads_started":1}
I20260928 14:40:33.626822 4526 tablet_service.cc:1514] Processing DeleteTablet for tablet abf27f74f1cb490a969142de9ac749dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:40:33 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:51856
I20260928 14:40:33.627466 4600 tablet_replica.cc:330] stopping tablet replica
I20260928 14:40:33.627609 4600 raft_consensus.cc:2229] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:40:33.627682 4600 raft_consensus.cc:2258] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:40:33.628038 4600 ts_tablet_manager.cc:1905] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:40:33.630494 4600 ts_tablet_manager.cc:1918] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:40:33.630601 4600 log.cc:1196] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal/wals/abf27f74f1cb490a969142de9ac749dc
I20260928 14:40:33.630831 4600 ts_tablet_manager.cc:1939] T abf27f74f1cb490a969142de9ac749dc P 0ec98f35ee1546dea30d1610662f24d7: Deleting consensus metadata
I20260928 14:40:33.631445 4389 catalog_manager.cc:4841] TS 0ec98f35ee1546dea30d1610662f24d7 (127.22.13.65:43987): tablet abf27f74f1cb490a969142de9ac749dc (table test-table [id=7f14648de444424c98129ed5b55870a1]) successfully deleted
W20260928 14:40:33.689569 4597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:33.689787 4597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:33.689813 4597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:33.690968 4597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:33.691006 4597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:33.691032 4597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:40:33.692467 4597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:33767
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=33537
--tserver_master_addrs=127.22.13.126:41033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:33.692675 4597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:33.692849 4597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:33.698698 4606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:33.698734 4608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:33.698724 4605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:33.698776 4597 server_base.cc:1034] running on GCE node
I20260928 14:40:33.699254 4597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:33.699503 4597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:33.700659 4597 hybrid_clock.cc:648] HybridClock initialized: now 1790606433700645 us; error 31 us; skew 500 ppm
Sep 28 14:40:33 dist-test-slave-wwwb krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606433, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:33.703166 4597 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:40:33.703585 4597 webserver.cc:460] Webserver started at http://127.22.13.66:33537/ using document root <none> and password file <none>
I20260928 14:40:33.703783 4597 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:33.703840 4597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:33.705255 4597 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:33.705826 4615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:33.706091 4597 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:33.706168 4597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "f285265e654e42bfa1bc49224de0ac2d"
format_stamp: "Formatted at 2026-09-28 14:40:33 on dist-test-slave-wwwb"
server_key: "55d221abe0a29a24f826ef3287c025df"
server_key_iv: "bb3cbf1d60ac0f6bf1eeb7371117cf36"
server_key_version: "encryptionkey@0"
I20260928 14:40:33.706501 4597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:33.730677 4597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:33.733119 4597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:33.733263 4597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:33.733458 4597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:33.733801 4597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:40:33.733837 4597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:33.733861 4597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:40:33.733909 4597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:33.739869 4597 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:33767
I20260928 14:40:33.739917 4728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:33767 every 8 connection(s)
I20260928 14:40:33.740271 4597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:40:33.745770 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 4597
I20260928 14:40:33.746106 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:38323
--local_ip_for_outbound_sockets=127.22.13.67
--tserver_master_addrs=127.22.13.126:41033
--webserver_port=36433
--webserver_interface=127.22.13.67
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:40:33 dist-test-slave-wwwb krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606433, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:33.750083 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.740986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:58155 (local address 127.22.13.126:41033)
0928 14:40:33.741055 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.741057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.741292 (+ 235us) server_negotiation.cc:407] Connection header received
0928 14:40:33.741771 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.741772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.741786 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.741842 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:33.742894 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.743453 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.744458 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.744658 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.744709 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.746994 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:33.746995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:33.746996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:33.747005 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:33.748444 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:33.748894 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:33.748896 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:33.748896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:33.748959 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:33.749269 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:33.749270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:33.749271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:33.749451 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:33.749522 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.749791 (+ 269us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.749906 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:40:33.750159 4731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.740984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:58155)
0928 14:40:33.741156 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.741194 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:33.741681 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:33.741924 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:33.741928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:33.742588 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:33.742804 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.742809 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.743540 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.743541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.744373 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.744376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.744732 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.744733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.744847 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.745326 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:33.745335 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:33.746881 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:33.748546 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:33.748554 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:33.748557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:33.748800 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:33.749070 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:33.749071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:33.749071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:33.749193 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:33.749520 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:33.749526 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:33.749657 (+ 131us) client_negotiation.cc:769] Sending connection context
0928 14:40:33.749744 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 14:40:33.749889 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260928 14:40:33.750720 4729 heartbeater.cc:347] Connected to a master server at 127.22.13.126:41033
I20260928 14:40:33.750819 4729 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:33.751039 4729 heartbeater.cc:511] Master 127.22.13.126:41033 requested a full tablet report, sending...
I20260928 14:40:33.751472 4403 ts_manager.cc:194] Registered new tserver with Master: f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767)
I20260928 14:40:33.751860 4403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:58155
W20260928 14:40:33.830849 4732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:33.831110 4732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:33.831153 4732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:33.832414 4732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:33.832482 4732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:33.832528 4732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:40:33.834076 4732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:38323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=36433
--tserver_master_addrs=127.22.13.126:41033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:33.834367 4732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:33.834612 4732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:33.839797 4738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:33.839887 4732 server_base.cc:1034] running on GCE node
W20260928 14:40:33.839810 4737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:33.839811 4740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:33.840233 4732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:33.840517 4732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:33.841681 4732 hybrid_clock.cc:648] HybridClock initialized: now 1790606433841677 us; error 54 us; skew 500 ppm
Sep 28 14:40:33 dist-test-slave-wwwb krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606433, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:33.844310 4732 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:40:33.844725 4732 webserver.cc:460] Webserver started at http://127.22.13.67:36433/ using document root <none> and password file <none>
I20260928 14:40:33.844923 4732 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:33.844975 4732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:33.846369 4732 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 14:40:33.846920 4747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:33.847147 4732 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:40:33.847237 4732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "22a55c393f224764b560f65f0526d943"
format_stamp: "Formatted at 2026-09-28 14:40:33 on dist-test-slave-wwwb"
server_key: "51524c1ecedb54c66fa7e7c3bc13115a"
server_key_iv: "39d55f54d1fe2458be655d4a088a4f11"
server_key_version: "encryptionkey@0"
I20260928 14:40:33.847597 4732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:33.857856 4732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:33.859833 4732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:33.859992 4732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:33.860248 4732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:33.860576 4732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:40:33.860631 4732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:33.860674 4732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:40:33.860702 4732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:33.866959 4732 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:38323
I20260928 14:40:33.867033 4860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:38323 every 8 connection(s)
I20260928 14:40:33.867389 4732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:40:33.872551 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 4732
Sep 28 14:40:33 dist-test-slave-wwwb krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606433, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:33.877246 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.868130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:57541 (local address 127.22.13.126:41033)
0928 14:40:33.868197 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.868199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.868424 (+ 225us) server_negotiation.cc:407] Connection header received
0928 14:40:33.868958 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.868959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.868971 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.869028 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:33.870029 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.870601 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.871619 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.871813 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.871920 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.874276 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:33.874277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:33.874277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:33.874285 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:33.875741 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:33.876159 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:33.876160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:33.876161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:33.876229 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:33.876575 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:33.876576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:33.876576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:33.876687 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:33.876765 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.877060 (+ 295us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.877144 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 14:40:33.877446 4863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.868125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:57541)
0928 14:40:33.868301 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.868338 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:33.868851 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:33.869105 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:33.869109 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:33.869687 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:33.869885 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.869890 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.870710 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.870711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.871502 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.871504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.871931 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.871932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.872065 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.872574 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:33.872585 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:33.874187 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:33.875820 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:33.875827 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:33.875829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:33.876059 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:33.876311 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:33.876331 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:33.876331 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:33.876491 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:33.876755 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:33.876762 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:33.876911 (+ 149us) client_negotiation.cc:769] Sending connection context
0928 14:40:33.877039 (+ 128us) client_negotiation.cc:240] Negotiation successful
0928 14:40:33.877220 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260928 14:40:33.877995 4861 heartbeater.cc:347] Connected to a master server at 127.22.13.126:41033
I20260928 14:40:33.878136 4861 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:33.878345 4861 heartbeater.cc:511] Master 127.22.13.126:41033 requested a full tablet report, sending...
I20260928 14:40:33.878803 4403 ts_manager.cc:194] Registered new tserver with Master: 22a55c393f224764b560f65f0526d943 (127.22.13.67:38323)
I20260928 14:40:33.879343 4403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:57541
I20260928 14:40:33.880105 4865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.873028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44114 (local address 127.22.13.126:41033)
0928 14:40:33.873136 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.873138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.873218 (+ 80us) server_negotiation.cc:407] Connection header received
0928 14:40:33.873298 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.873299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.873310 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.873388 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:33.874022 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.874698 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.875570 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.875911 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.875984 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.877180 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:33.877181 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:33.877181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:33.877188 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:33.878902 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:33.879283 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:33.879283 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:33.879283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:33.879341 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:33.879544 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:33.879545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:33.879546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:33.879677 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:33.879820 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.879906 (+ 86us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.880022 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":53,"threads_started":1}
I20260928 14:40:33.880836 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:40:33.883862 4865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.881197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44126 (local address 127.22.13.126:41033)
0928 14:40:33.881254 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.881256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.881386 (+ 130us) server_negotiation.cc:407] Connection header received
0928 14:40:33.881472 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.881473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.881512 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.881593 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:40:33.881741 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.882280 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.883043 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.883269 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.883344 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.883518 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:40:33.883621 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:40:33.883710 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.883760 (+ 50us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.883785 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:40:33.884121 4403 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:40:33.885643 4403 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:40:33.887956 4526 tablet_service.cc:1467] Processing CreateTablet for tablet a5fcdc4443d9412fb49b02ce1437b553 (DEFAULT_TABLE table=test-table [id=8279992c3e23464bbe5dd5254c4da28b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:40:33.888173 4526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5fcdc4443d9412fb49b02ce1437b553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:33.889662 4483 tablet_bootstrap.cc:492] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7: Bootstrap starting.
I20260928 14:40:33.890156 4483 tablet_bootstrap.cc:654] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:33.891079 4483 tablet_bootstrap.cc:492] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7: No bootstrap required, opened a new log
I20260928 14:40:33.891124 4483 ts_tablet_manager.cc:1397] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:33.891296 4483 raft_consensus.cc:348] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.891368 4483 raft_consensus.cc:374] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:33.891412 4483 raft_consensus.cc:729] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec98f35ee1546dea30d1610662f24d7, State: Initialized, Role: FOLLOWER
I20260928 14:40:33.891458 4483 consensus_queue.cc:260] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.891566 4483 ts_tablet_manager.cc:1428] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:33.891633 4592 heartbeater.cc:503] Master 127.22.13.126:41033 was elected leader, sending a full tablet report...
I20260928 14:40:33.891718 4867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.887732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:38323 (local address 127.0.0.1:50680)
0928 14:40:33.887914 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.887926 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:33.887979 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:33.888366 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:33.888367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:33.888370 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:33.888421 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.888423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.889515 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.889516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.890517 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.890519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.891425 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.891427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.891495 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.891517 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 14:40:33.891591 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 14:40:33.891644 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":2,"thread_start_us":133,"threads_started":1}
I20260928 14:40:33.892007 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.887996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50680 (local address 127.22.13.67:38323)
0928 14:40:33.888178 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.888183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.888193 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:40:33.888236 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.888238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.888270 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.888353 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:33.888479 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.889414 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.890603 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.891326 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.891422 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.891485 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.891879 (+ 394us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.891925 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":75,"threads_started":1}
I20260928 14:40:33.892807 4795 tablet_service.cc:1467] Processing CreateTablet for tablet a5fcdc4443d9412fb49b02ce1437b553 (DEFAULT_TABLE table=test-table [id=8279992c3e23464bbe5dd5254c4da28b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:40:33.893169 4795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5fcdc4443d9412fb49b02ce1437b553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:33.893632 4598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.887691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:33767 (local address 127.0.0.1:34918)
0928 14:40:33.887736 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.887748 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:33.887784 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:33.889983 (+ 2199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:33.889984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:33.889986 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:33.890022 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.890024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.891305 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.891306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.892400 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.892403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.893410 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.893411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.893486 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.893491 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:40:33.893540 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 14:40:33.893582 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":1}
I20260928 14:40:33.893714 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.887880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34918 (local address 127.22.13.66:33767)
0928 14:40:33.889771 (+ 1891us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.889776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.889788 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:40:33.889835 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.889837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.889856 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.889985 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:33.890100 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.891180 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.892504 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.893252 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.893362 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.893421 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.893578 (+ 157us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.893622 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1852,"thread_start_us":98,"threads_started":1}
I20260928 14:40:33.894834 4663 tablet_service.cc:1467] Processing CreateTablet for tablet a5fcdc4443d9412fb49b02ce1437b553 (DEFAULT_TABLE table=test-table [id=8279992c3e23464bbe5dd5254c4da28b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:40:33.895184 4663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5fcdc4443d9412fb49b02ce1437b553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:33.896871 4871 tablet_bootstrap.cc:492] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943: Bootstrap starting.
I20260928 14:40:33.897459 4871 tablet_bootstrap.cc:654] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:33.898018 4871 log.cc:824] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:33.898797 4871 tablet_bootstrap.cc:492] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943: No bootstrap required, opened a new log
I20260928 14:40:33.898881 4871 ts_tablet_manager.cc:1397] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 14:40:33.900435 4871 raft_consensus.cc:348] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.900596 4871 raft_consensus.cc:374] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:33.900642 4871 raft_consensus.cc:729] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22a55c393f224764b560f65f0526d943, State: Initialized, Role: FOLLOWER
I20260928 14:40:33.900786 4871 consensus_queue.cc:260] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.901108 4871 ts_tablet_manager.cc:1428] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 14:40:33.901130 4861 heartbeater.cc:503] Master 127.22.13.126:41033 was elected leader, sending a full tablet report...
I20260928 14:40:33.901214 4873 tablet_bootstrap.cc:492] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d: Bootstrap starting.
I20260928 14:40:33.901942 4873 tablet_bootstrap.cc:654] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:33.902503 4873 log.cc:824] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:33.903189 4873 tablet_bootstrap.cc:492] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d: No bootstrap required, opened a new log
I20260928 14:40:33.903246 4873 ts_tablet_manager.cc:1397] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260928 14:40:33.904779 4873 raft_consensus.cc:348] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.904908 4873 raft_consensus.cc:374] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:33.904929 4873 raft_consensus.cc:729] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f285265e654e42bfa1bc49224de0ac2d, State: Initialized, Role: FOLLOWER
I20260928 14:40:33.905011 4873 consensus_queue.cc:260] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.905221 4873 ts_tablet_manager.cc:1428] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:40:33.905262 4729 heartbeater.cc:503] Master 127.22.13.126:41033 was elected leader, sending a full tablet report...
I20260928 14:40:33.951486 4874 raft_consensus.cc:482] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:40:33.951620 4874 raft_consensus.cc:504] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.952095 4874 leader_election.cc:288] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767), 0ec98f35ee1546dea30d1610662f24d7 (127.22.13.65:43987)
I20260928 14:40:33.954928 4863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.952259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:33767 (local address 127.22.13.67:54601)
0928 14:40:33.952312 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.952323 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:33.952373 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:33.952585 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:33.952586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:33.952589 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:33.952664 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.952666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.953314 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.953315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.954062 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.954064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.954639 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.954640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.954725 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.954728 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:40:33.954793 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 14:40:33.954839 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260928 14:40:33.954913 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.952393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:54601 (local address 127.22.13.66:33767)
0928 14:40:33.952442 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.952445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.952455 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:40:33.952492 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.952493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.952500 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.952579 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:33.952741 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.953225 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.954162 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.954553 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.954643 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.954652 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.954812 (+ 160us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.954849 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:40:33.955228 4599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.952373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:32801 (local address 127.22.13.65:43987)
0928 14:40:33.952437 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.952440 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.952551 (+ 111us) server_negotiation.cc:407] Connection header received
0928 14:40:33.952653 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.952654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.952683 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.952749 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:33.952917 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.953427 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.954421 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.954842 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.954931 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.955009 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.955086 (+ 77us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.955142 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 14:40:33.955233 4877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.952294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:43987 (local address 127.22.13.67:32801)
0928 14:40:33.952488 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.952495 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:33.952560 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:33.952769 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:33.952771 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:33.952773 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:33.952844 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.952846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.953517 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.953518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.954331 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:33.954333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.954931 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.954932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:33.954996 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.955001 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:40:33.955076 (+ 75us) client_negotiation.cc:240] Negotiation successful
0928 14:40:33.955124 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":1,"thread_start_us":116,"threads_started":1}
I20260928 14:40:33.955370 4683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5fcdc4443d9412fb49b02ce1437b553" candidate_uuid: "22a55c393f224764b560f65f0526d943" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f285265e654e42bfa1bc49224de0ac2d" is_pre_election: true
I20260928 14:40:33.955546 4683 raft_consensus.cc:2454] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22a55c393f224764b560f65f0526d943 in term 0.
I20260928 14:40:33.955674 4546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5fcdc4443d9412fb49b02ce1437b553" candidate_uuid: "22a55c393f224764b560f65f0526d943" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec98f35ee1546dea30d1610662f24d7" is_pre_election: true
I20260928 14:40:33.955798 4749 leader_election.cc:302] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22a55c393f224764b560f65f0526d943, f285265e654e42bfa1bc49224de0ac2d; no voters:
I20260928 14:40:33.955847 4546 raft_consensus.cc:2454] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22a55c393f224764b560f65f0526d943 in term 0.
I20260928 14:40:33.955933 4874 raft_consensus.cc:2781] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:40:33.956027 4874 raft_consensus.cc:482] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:40:33.956051 4874 raft_consensus.cc:3037] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:33.956760 4874 raft_consensus.cc:504] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.956904 4874 leader_election.cc:288] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [CANDIDATE]: Term 1 election: Requested vote from peers f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767), 0ec98f35ee1546dea30d1610662f24d7 (127.22.13.65:43987)
I20260928 14:40:33.957062 4683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5fcdc4443d9412fb49b02ce1437b553" candidate_uuid: "22a55c393f224764b560f65f0526d943" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f285265e654e42bfa1bc49224de0ac2d"
I20260928 14:40:33.957075 4546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5fcdc4443d9412fb49b02ce1437b553" candidate_uuid: "22a55c393f224764b560f65f0526d943" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec98f35ee1546dea30d1610662f24d7"
I20260928 14:40:33.957150 4683 raft_consensus.cc:3037] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:33.957173 4546 raft_consensus.cc:3037] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:33.957854 4546 raft_consensus.cc:2454] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22a55c393f224764b560f65f0526d943 in term 1.
I20260928 14:40:33.957856 4683 raft_consensus.cc:2454] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22a55c393f224764b560f65f0526d943 in term 1.
I20260928 14:40:33.958067 4750 leader_election.cc:302] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec98f35ee1546dea30d1610662f24d7, 22a55c393f224764b560f65f0526d943; no voters:
I20260928 14:40:33.958169 4874 raft_consensus.cc:2781] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:40:33.958320 4874 raft_consensus.cc:686] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [term 1 LEADER]: Becoming Leader. State: Replica: 22a55c393f224764b560f65f0526d943, State: Running, Role: LEADER
I20260928 14:40:33.958431 4874 consensus_queue.cc:237] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:33.959347 4403 catalog_manager.cc:5495] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22a55c393f224764b560f65f0526d943 (127.22.13.67). New cstate: current_term: 1 leader_uuid: "22a55c393f224764b560f65f0526d943" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } health_report { overall_health: UNKNOWN } } }
I20260928 14:40:33.966100 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.963559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50682 (local address 127.22.13.67:38323)
0928 14:40:33.963609 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:33.963611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:33.963618 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:40:33.963671 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:33.963672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:33.963686 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:33.963746 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:40:33.963889 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.964466 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.965350 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:33.965530 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:33.965590 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:33.965734 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:40:33.965834 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:40:33.965896 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:33.965969 (+ 73us) server_negotiation.cc:299] Negotiation successful
0928 14:40:33.965993 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 14:40:33.967866 4683 raft_consensus.cc:1261] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d [term 1 FOLLOWER]: Refusing update from remote peer 22a55c393f224764b560f65f0526d943: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:40:33.968227 4546 raft_consensus.cc:1261] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Refusing update from remote peer 22a55c393f224764b560f65f0526d943: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:40:33.968246 4878 consensus_queue.cc:1035] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:40:33.968523 4878 consensus_queue.cc:1035] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:40:33.971335 4881 mvcc.cc:205] Tried to move back new op lower bound from 7334323953529892864 to 7334323953494618112. Current Snapshot: MvccSnapshot[applied={T|T < 7334323953529892864}]
I20260928 14:40:33.972051 4884 mvcc.cc:205] Tried to move back new op lower bound from 7334323953529892864 to 7334323953494618112. Current Snapshot: MvccSnapshot[applied={T|T < 7334323953529892864}]
I20260928 14:40:33.974046 4403 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:40:33.974169 4403 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:40:33.974915 4403 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee: Sending DeleteTablet for 3 replicas of tablet a5fcdc4443d9412fb49b02ce1437b553
I20260928 14:40:33.975237 4663 tablet_service.cc:1514] Processing DeleteTablet for tablet a5fcdc4443d9412fb49b02ce1437b553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:40:33 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:34918
I20260928 14:40:33.975296 4526 tablet_service.cc:1514] Processing DeleteTablet for tablet a5fcdc4443d9412fb49b02ce1437b553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:40:33 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:51856
I20260928 14:40:33.975473 4892 tablet_replica.cc:330] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d: stopping tablet replica
I20260928 14:40:33.975466 4795 tablet_service.cc:1514] Processing DeleteTablet for tablet a5fcdc4443d9412fb49b02ce1437b553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:40:33 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:50680
I20260928 14:40:33.975585 4892 raft_consensus.cc:2229] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:40:33.975637 4600 tablet_replica.cc:330] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7: stopping tablet replica
I20260928 14:40:33.975666 4893 tablet_replica.cc:330] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943: stopping tablet replica
I20260928 14:40:33.975698 4600 raft_consensus.cc:2229] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:40:33.975746 4600 raft_consensus.cc:2258] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:40:33.975766 4893 raft_consensus.cc:2229] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:40:33.975804 4892 raft_consensus.cc:2258] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:40:33.975881 4893 raft_consensus.cc:2258] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:40:33.975983 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 4441
I20260928 14:40:33.976776 4892 ts_tablet_manager.cc:1905] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:40:33.976869 4893 ts_tablet_manager.cc:1905] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:40:33.978556 4893 ts_tablet_manager.cc:1918] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:40:33.978636 4893 log.cc:1196] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/a5fcdc4443d9412fb49b02ce1437b553
I20260928 14:40:33.978792 4892 ts_tablet_manager.cc:1918] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:40:33.978863 4892 log.cc:1196] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/a5fcdc4443d9412fb49b02ce1437b553
I20260928 14:40:33.978931 4893 ts_tablet_manager.cc:1939] T a5fcdc4443d9412fb49b02ce1437b553 P 22a55c393f224764b560f65f0526d943: Deleting consensus metadata
I20260928 14:40:33.979151 4892 ts_tablet_manager.cc:1939] T a5fcdc4443d9412fb49b02ce1437b553 P f285265e654e42bfa1bc49224de0ac2d: Deleting consensus metadata
I20260928 14:40:33.979423 4387 catalog_manager.cc:4841] TS 22a55c393f224764b560f65f0526d943 (127.22.13.67:38323): tablet a5fcdc4443d9412fb49b02ce1437b553 (table test-table [id=8279992c3e23464bbe5dd5254c4da28b]) successfully deleted
I20260928 14:40:33.979729 4388 catalog_manager.cc:4841] TS f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767): tablet a5fcdc4443d9412fb49b02ce1437b553 (table test-table [id=8279992c3e23464bbe5dd5254c4da28b]) successfully deleted
W20260928 14:40:33.983186 4389 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:43987 (error 108)
I20260928 14:40:33.983363 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 4597
I20260928 14:40:33.984036 4598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:33.983599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:43987 (local address 127.0.0.1:51866)
0928 14:40:33.983947 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:33.984000 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322}
W20260928 14:40:33.984179 4389 catalog_manager.cc:4584] TS 0ec98f35ee1546dea30d1610662f24d7 (127.22.13.65:43987): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a5fcdc4443d9412fb49b02ce1437b553: Network error: Client connection negotiation failed: client connection to 127.22.13.65:43987: connect: Connection refused (error 111)
I20260928 14:40:33.987991 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 4732
I20260928 14:40:33.992785 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 4370
I20260928 14:40:33.998554 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41033
--webserver_interface=127.22.13.126
--webserver_port=43083
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:41033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:34.082216 4895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:34.082490 4895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:34.082531 4895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:34.083838 4895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:40:34.083920 4895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:34.083945 4895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:34.083978 4895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:40:34.084025 4895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:40:34.085640 4895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:41033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=43083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:34.086018 4895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:34.086272 4895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:34.091172 4901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:34.091166 4900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:34.091183 4903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:34.091333 4895 server_base.cc:1034] running on GCE node
I20260928 14:40:34.091712 4895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:34.092002 4895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:34.093163 4895 hybrid_clock.cc:648] HybridClock initialized: now 1790606434093148 us; error 46 us; skew 500 ppm
Sep 28 14:40:34 dist-test-slave-wwwb krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606434, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:34.095829 4895 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:40:34.096333 4895 webserver.cc:460] Webserver started at http://127.22.13.126:43083/ using document root <none> and password file <none>
I20260928 14:40:34.096591 4895 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:34.096638 4895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:34.098042 4895 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:34.098729 4910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:34.098903 4895 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:40:34.098982 4895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "8930beb078004e3c99c96a45245e53ee"
format_stamp: "Formatted at 2026-09-28 14:40:32 on dist-test-slave-wwwb"
server_key: "b6537596dea83803ddd1c4dbb6bce064"
server_key_iv: "f7e18751bf98d7ce03e7a47fc1c6f8e6"
server_key_version: "encryptionkey@0"
I20260928 14:40:34.099324 4895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:34.109163 4895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:34.111088 4895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:34.111202 4895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:34.115054 4895 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:41033
I20260928 14:40:34.115088 4962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:41033 every 8 connection(s)
I20260928 14:40:34.115490 4895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:40:34.117118 4963 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:40:34.117817 4963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee: Bootstrap starting.
I20260928 14:40:34.123605 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 4895
I20260928 14:40:34.123916 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:43987
--local_ip_for_outbound_sockets=127.22.13.65
--tserver_master_addrs=127.22.13.126:41033
--webserver_port=38769
--webserver_interface=127.22.13.65
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:40:34.126770 4963 log.cc:824] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:34.129215 4963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:40:34.129550 4963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee: Bootstrap complete.
I20260928 14:40:34.131932 4963 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } }
I20260928 14:40:34.132202 4963 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8930beb078004e3c99c96a45245e53ee, State: Initialized, Role: FOLLOWER
I20260928 14:40:34.132405 4963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } }
I20260928 14:40:34.132488 4963 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:40:34.132542 4963 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:40:34.132593 4963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 2 FOLLOWER]: Advancing to term 3
I20260928 14:40:34.133675 4963 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } }
I20260928 14:40:34.133808 4963 leader_election.cc:302] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8930beb078004e3c99c96a45245e53ee; no voters:
I20260928 14:40:34.134065 4963 leader_election.cc:288] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [CANDIDATE]: Term 3 election: Requested vote from peers
I20260928 14:40:34.134109 4967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 3 FOLLOWER]: Leader election won for term 3
I20260928 14:40:34.134222 4967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [term 3 LEADER]: Becoming Leader. State: Replica: 8930beb078004e3c99c96a45245e53ee, State: Running, Role: LEADER
I20260928 14:40:34.134377 4967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } }
I20260928 14:40:34.134382 4963 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:40:34.134814 4969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8930beb078004e3c99c96a45245e53ee. Latest consensus state: current_term: 3 leader_uuid: "8930beb078004e3c99c96a45245e53ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } } }
I20260928 14:40:34.134799 4968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8930beb078004e3c99c96a45245e53ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8930beb078004e3c99c96a45245e53ee" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41033 } } }
I20260928 14:40:34.134898 4969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:34.134902 4968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8930beb078004e3c99c96a45245e53ee [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:34.135154 4971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:40:34.135991 4971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:40:34.136094 4971 catalog_manager.cc:1259] Loaded cluster ID: a2039240fb5e40468033b8baab06fbef
I20260928 14:40:34.136127 4971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260928 14:40:34.136442 4971 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f47571739ad google::LogMessage::Fail() at ??:0
@ 0x7f475717688a google::LogMessage::SendToLog() at ??:0
@ 0x7f47571734de google::LogMessage::Flush() at ??:0
@ 0x7f4757174a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f475f0940f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f475f09d8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f47590410d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4759036954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4759e9a6db start_thread at ??:0
@ 0x7f4755c7161f clone at ??:0
W20260928 14:40:34.223188 4965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:34.223405 4965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:34.223433 4965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:34.224654 4965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:34.224704 4965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:34.224733 4965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:40:34.226347 4965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:43987
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=38769
--tserver_master_addrs=127.22.13.126:41033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:34.226614 4965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:34.226866 4965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:34.232081 5000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:34.232089 4999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:34.232461 5002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:34.233127 4965 server_base.cc:1034] running on GCE node
I20260928 14:40:34.233395 4965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:34.233705 4965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:34.234907 4965 hybrid_clock.cc:648] HybridClock initialized: now 1790606434234867 us; error 52 us; skew 500 ppm
Sep 28 14:40:34 dist-test-slave-wwwb krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606434, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:34.238158 4965 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:40:34.238799 4965 webserver.cc:460] Webserver started at http://127.22.13.65:38769/ using document root <none> and password file <none>
I20260928 14:40:34.239082 4965 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:34.239158 4965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:34.240984 4965 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:40:34.241823 5009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:34.242043 4965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:34.242146 4965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "0ec98f35ee1546dea30d1610662f24d7"
format_stamp: "Formatted at 2026-09-28 14:40:32 on dist-test-slave-wwwb"
server_key: "0b75f8158b42cca2d8d8c956d106a05f"
server_key_iv: "309da32ea8630b115f1c7c346b9716d5"
server_key_version: "encryptionkey@0"
I20260928 14:40:34.242630 4965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:34.258643 4965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:34.261303 4965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:34.261480 4965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:34.261787 4965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:34.262337 5016 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 14:40:34.263823 4965 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 14:40:34.263890 4965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 14:40:34.263926 4965 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 14:40:34.264861 4965 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 14:40:34.264909 4965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260928 14:40:34.264988 5016 tablet_bootstrap.cc:492] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7: Bootstrap starting.
I20260928 14:40:34.272213 4965 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:43987
I20260928 14:40:34.272682 4965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:40:34.273131 5123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:43987 every 8 connection(s)
I20260928 14:40:34.276036 5016 log.cc:824] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:34.278473 5016 tablet_bootstrap.cc:492] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:40:34.278774 5016 tablet_bootstrap.cc:492] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7: Bootstrap complete.
I20260928 14:40:34.279624 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 4965
I20260928 14:40:34.279845 5016 ts_tablet_manager.cc:1397] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260928 14:40:34.279927 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:33767
--local_ip_for_outbound_sockets=127.22.13.66
--tserver_master_addrs=127.22.13.126:41033
--webserver_port=33537
--webserver_interface=127.22.13.66
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:40:34.281615 5016 raft_consensus.cc:348] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:34.281890 5016 raft_consensus.cc:729] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec98f35ee1546dea30d1610662f24d7, State: Initialized, Role: FOLLOWER
I20260928 14:40:34.282069 5016 consensus_queue.cc:260] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:34.282423 5016 ts_tablet_manager.cc:1428] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
W20260928 14:40:34.336103 5126 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:40:34.273540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:41631)
0928 14:40:34.273752 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:34.273803 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:34.274493 (+ 690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:34.335885 (+ 61392us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":93,"threads_started":1}
W20260928 14:40:34.336370 5124 heartbeater.cc:644] Failed to heartbeat to 127.22.13.126:41033 (0 consecutive failures): Network error: Failed to ping master at 127.22.13.126:41033: Client connection negotiation failed: client connection to 127.22.13.126:41033: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 14:40:34.336771 5126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:34.336649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:38085)
0928 14:40:34.336707 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:34.336751 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 14:40:34.337244 5126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:34.337044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:38529)
0928 14:40:34.337193 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:34.337218 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260928 14:40:34.337364 5124 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:40:34.368851 5128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:34.369125 5128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:34.369172 5128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:34.370522 5128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:34.370600 5128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:34.370649 5128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:40:34.372392 5128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:33767
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=33537
--tserver_master_addrs=127.22.13.126:41033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:34.372684 5128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:34.372936 5128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:34.378261 5137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:34.378360 5135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:34.378247 5134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:34.378551 5128 server_base.cc:1034] running on GCE node
I20260928 14:40:34.378744 5128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:34.378969 5128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:34.380111 5128 hybrid_clock.cc:648] HybridClock initialized: now 1790606434380098 us; error 29 us; skew 500 ppm
Sep 28 14:40:34 dist-test-slave-wwwb krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606434, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:34.382645 5128 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:40:34.383169 5128 webserver.cc:460] Webserver started at http://127.22.13.66:33537/ using document root <none> and password file <none>
I20260928 14:40:34.383425 5128 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:34.383476 5128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:34.384891 5128 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:34.385584 5144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:34.385774 5128 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:34.385874 5128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "f285265e654e42bfa1bc49224de0ac2d"
format_stamp: "Formatted at 2026-09-28 14:40:33 on dist-test-slave-wwwb"
server_key: "55d221abe0a29a24f826ef3287c025df"
server_key_iv: "bb3cbf1d60ac0f6bf1eeb7371117cf36"
server_key_version: "encryptionkey@0"
I20260928 14:40:34.386319 5128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:34.396203 5128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:34.398136 5128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:34.398303 5128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:34.398545 5128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:34.398856 5128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:40:34.398888 5128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:34.398911 5128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:40:34.398958 5128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:34.405159 5128 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:33767
I20260928 14:40:34.405233 5257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:33767 every 8 connection(s)
I20260928 14:40:34.405584 5128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:40:34.405773 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 5128
I20260928 14:40:34.406092 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:38323
--local_ip_for_outbound_sockets=127.22.13.67
--tserver_master_addrs=127.22.13.126:41033
--webserver_port=36433
--webserver_interface=127.22.13.67
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:40:34.406916 5260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:34.406408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:59273)
0928 14:40:34.406596 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:34.406715 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
W20260928 14:40:34.407440 5258 heartbeater.cc:644] Failed to heartbeat to 127.22.13.126:41033 (0 consecutive failures): Network error: Failed to ping master at 127.22.13.126:41033: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
I20260928 14:40:34.407796 5260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:34.407673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:46925)
0928 14:40:34.407727 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:34.407765 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260928 14:40:34.408159 5260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:34.408067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:43209)
0928 14:40:34.408114 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:34.408139 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 14:40:34.408300 5258 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:40:34.489827 5261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:34.490073 5261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:34.490136 5261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:34.491353 5261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:34.491400 5261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:34.491428 5261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:40:34.492892 5261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:38323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=36433
--tserver_master_addrs=127.22.13.126:41033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:34.493108 5261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:34.493302 5261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:34.498371 5266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:34.498365 5267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:34.498406 5269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:34.498605 5261 server_base.cc:1034] running on GCE node
I20260928 14:40:34.498799 5261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:34.499114 5261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:34.500272 5261 hybrid_clock.cc:648] HybridClock initialized: now 1790606434500260 us; error 31 us; skew 500 ppm
Sep 28 14:40:34 dist-test-slave-wwwb krb5kdc[3822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606434, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:34.502826 5261 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:40:34.503245 5261 webserver.cc:460] Webserver started at http://127.22.13.67:36433/ using document root <none> and password file <none>
I20260928 14:40:34.503443 5261 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:34.503484 5261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:34.504806 5261 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:40:34.505400 5276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:34.505558 5261 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:40:34.505645 5261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "22a55c393f224764b560f65f0526d943"
format_stamp: "Formatted at 2026-09-28 14:40:33 on dist-test-slave-wwwb"
server_key: "51524c1ecedb54c66fa7e7c3bc13115a"
server_key_iv: "39d55f54d1fe2458be655d4a088a4f11"
server_key_version: "encryptionkey@0"
I20260928 14:40:34.506090 5261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:34.523952 5261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:34.525861 5261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:34.526029 5261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:34.526300 5261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:34.526628 5261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:40:34.526667 5261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:34.526691 5261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:40:34.526738 5261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:34.532519 5261 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:38323
I20260928 14:40:34.532622 5389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:38323 every 8 connection(s)
I20260928 14:40:34.532963 5261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:40:34.534144 5392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:34.533671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:36289)
0928 14:40:34.533842 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:34.533925 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
W20260928 14:40:34.534574 5390 heartbeater.cc:644] Failed to heartbeat to 127.22.13.126:41033 (0 consecutive failures): Network error: Failed to ping master at 127.22.13.126:41033: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
I20260928 14:40:34.534897 5392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:34.534798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:53993)
0928 14:40:34.534847 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:34.534878 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260928 14:40:34.535214 5392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:34.535126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:50391)
0928 14:40:34.535171 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:34.535194 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260928 14:40:34.535355 5390 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 14:40:34.542181 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 5261
I20260928 14:40:35.338153 5394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:35.337827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:60727)
0928 14:40:35.338056 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:35.338102 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":65,"threads_started":1}
I20260928 14:40:35.408910 5395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:35.408634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:40971)
0928 14:40:35.408769 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:35.408810 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260928 14:40:35.505287 5396 raft_consensus.cc:482] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:40:35.505470 5396 raft_consensus.cc:504] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:35.505891 5396 leader_election.cc:288] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767), 22a55c393f224764b560f65f0526d943 (127.22.13.67:38323)
Sep 28 14:40:35 dist-test-slave-wwwb krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606434, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.67@KRBTEST.COM
Sep 28 14:40:35 dist-test-slave-wwwb krb5kdc[3822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606434, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.66@KRBTEST.COM
I20260928 14:40:35.518934 5394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:35.506078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:38323 (local address 127.22.13.65:51059)
0928 14:40:35.506224 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:35.506237 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:35.506299 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:35.507428 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:35.507432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:35.507872 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:35.508196 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:35.508203 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:35.510052 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:35.510053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:35.511299 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:35.511302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:35.512068 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:35.512069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:35.512183 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:35.512853 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:35.512874 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:35.514663 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:35.516914 (+ 2251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:35.516921 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:35.516923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:35.517369 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:35.517688 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:35.517688 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:35.517689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:35.517862 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:35.518407 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:35.518414 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:35.518559 (+ 145us) client_negotiation.cc:769] Sending connection context
0928 14:40:35.518684 (+ 125us) client_negotiation.cc:240] Negotiation successful
0928 14:40:35.518857 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":35}
I20260928 14:40:35.519011 5399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:35.506402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:49431 (local address 127.22.13.66:33767)
0928 14:40:35.506544 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:35.506555 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:35.507187 (+ 632us) server_negotiation.cc:407] Connection header received
0928 14:40:35.507731 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:35.507734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:35.507775 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:35.507934 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:35.508483 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:35.509790 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:35.512148 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:35.512705 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:35.512820 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:35.515153 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:35.515159 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:35.515159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:35.515175 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:35.517023 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:35.517479 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:35.517483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:35.517483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:35.517606 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:35.517963 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:35.517963 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:35.517964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:35.518307 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:35.518452 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:35.518734 (+ 282us) server_negotiation.cc:299] Negotiation successful
0928 14:40:35.518896 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":57,"threads_started":1}
I20260928 14:40:35.519049 5397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:35.506096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:33767 (local address 127.22.13.65:49431)
0928 14:40:35.507083 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:35.507096 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:35.507175 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:35.508039 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:35.508040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:35.508274 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:35.508336 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:35.508340 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:35.509918 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:35.509920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:35.511299 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:35.511302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:35.512798 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:35.512799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:35.512857 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:35.513199 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:35.513203 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:35.515037 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:35.517104 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:35.517105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:35.517106 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:35.517398 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:35.517688 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:35.517688 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:35.517689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:35.517862 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:35.518448 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:35.518449 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:35.518556 (+ 107us) client_negotiation.cc:769] Sending connection context
0928 14:40:35.518682 (+ 126us) client_negotiation.cc:240] Negotiation successful
0928 14:40:35.518949 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":968,"mutex_wait_us":165,"thread_start_us":66,"threads_started":1}
I20260928 14:40:35.519544 5398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:35.506215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:51059 (local address 127.22.13.67:38323)
0928 14:40:35.506563 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:35.506571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:35.506591 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:40:35.507123 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:35.507125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:35.507215 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:35.507357 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:35.508326 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:35.509863 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:35.511497 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:35.511953 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:35.512051 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:35.514807 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:35.514812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:35.514813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:35.514829 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:35.516813 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:35.517480 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:35.517483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:35.517483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:35.517606 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:35.517962 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:35.517963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:35.517964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:35.518307 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:35.518403 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:35.519300 (+ 897us) server_negotiation.cc:299] Negotiation successful
0928 14:40:35.519440 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":82,"threads_started":1}
W20260928 14:40:35.519845 5011 leader_election.cc:341] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767): Not found: Tablet not found: a5fcdc4443d9412fb49b02ce1437b553
W20260928 14:40:35.520237 5010 leader_election.cc:341] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22a55c393f224764b560f65f0526d943 (127.22.13.67:38323): Not found: Tablet not found: a5fcdc4443d9412fb49b02ce1437b553
I20260928 14:40:35.520289 5010 leader_election.cc:302] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ec98f35ee1546dea30d1610662f24d7; no voters: 22a55c393f224764b560f65f0526d943, f285265e654e42bfa1bc49224de0ac2d
I20260928 14:40:35.520391 5396 raft_consensus.cc:2726] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 14:40:35.535974 5400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:35.535714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:45769)
0928 14:40:35.535877 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:35.535931 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260928 14:40:36.338800 5401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:36.338567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:40071)
0928 14:40:36.338709 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:36.338751 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":64,"threads_started":1}
I20260928 14:40:36.409693 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:36.409444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:47815)
0928 14:40:36.409598 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:36.409644 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
I20260928 14:40:36.536931 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:36.536622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:33519)
0928 14:40:36.536760 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:36.536803 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260928 14:40:37.339488 5404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:37.339285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:33273)
0928 14:40:37.339409 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:37.339440 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
I20260928 14:40:37.410416 5405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:37.410206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:48925)
0928 14:40:37.410331 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:37.410373 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260928 14:40:37.537810 5406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:37.537577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:58623)
0928 14:40:37.537726 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:37.537768 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260928 14:40:37.700892 5407 raft_consensus.cc:482] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:40:37.701030 5407 raft_consensus.cc:504] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:37.701279 5407 leader_election.cc:288] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767), 22a55c393f224764b560f65f0526d943 (127.22.13.67:38323)
W20260928 14:40:37.701682 5011 leader_election.cc:341] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767): Not found: Tablet not found: a5fcdc4443d9412fb49b02ce1437b553
W20260928 14:40:37.701774 5010 leader_election.cc:341] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22a55c393f224764b560f65f0526d943 (127.22.13.67:38323): Not found: Tablet not found: a5fcdc4443d9412fb49b02ce1437b553
I20260928 14:40:37.701849 5011 leader_election.cc:302] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ec98f35ee1546dea30d1610662f24d7; no voters: 22a55c393f224764b560f65f0526d943, f285265e654e42bfa1bc49224de0ac2d
I20260928 14:40:37.701948 5407 raft_consensus.cc:2726] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 14:40:38.340394 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:38.340190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:36671)
0928 14:40:38.340317 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:38.340348 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260928 14:40:38.411149 5409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:38.410917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:47507)
0928 14:40:38.411056 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:38.411099 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260928 14:40:38.538564 5410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:38.538328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:36015)
0928 14:40:38.538466 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:38.538515 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":58,"threads_started":1}
I20260928 14:40:39.341197 5411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:39.340929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:58629)
0928 14:40:39.341060 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:39.341120 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":58,"threads_started":1}
I20260928 14:40:39.412073 5412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:39.411754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:53305)
0928 14:40:39.411976 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:39.412024 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":74,"threads_started":1}
I20260928 14:40:39.539299 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:39.539072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:46421)
0928 14:40:39.539205 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:39.539253 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260928 14:40:39.567834 5414 raft_consensus.cc:482] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:40:39.567931 5414 raft_consensus.cc:504] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:39.568118 5414 leader_election.cc:288] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767), 22a55c393f224764b560f65f0526d943 (127.22.13.67:38323)
W20260928 14:40:39.568575 5011 leader_election.cc:341] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767): Not found: Tablet not found: a5fcdc4443d9412fb49b02ce1437b553
W20260928 14:40:39.568661 5010 leader_election.cc:341] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22a55c393f224764b560f65f0526d943 (127.22.13.67:38323): Not found: Tablet not found: a5fcdc4443d9412fb49b02ce1437b553
I20260928 14:40:39.568703 5010 leader_election.cc:302] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ec98f35ee1546dea30d1610662f24d7; no voters: 22a55c393f224764b560f65f0526d943, f285265e654e42bfa1bc49224de0ac2d
I20260928 14:40:39.568789 5414 raft_consensus.cc:2726] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 14:40:40.342141 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:40.341843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:57777)
0928 14:40:40.342046 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:40.342092 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":74,"threads_started":1}
I20260928 14:40:40.412989 5416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:40.412722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:52827)
0928 14:40:40.412893 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:40.412939 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":93,"threads_started":1}
I20260928 14:40:40.540084 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:40.539850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:39681)
0928 14:40:40.539993 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:40.540037 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
I20260928 14:40:41.223886 5418 raft_consensus.cc:482] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:40:41.223994 5418 raft_consensus.cc:504] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:41.224229 5418 leader_election.cc:288] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767), 22a55c393f224764b560f65f0526d943 (127.22.13.67:38323)
W20260928 14:40:41.224624 5011 leader_election.cc:341] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767): Not found: Tablet not found: a5fcdc4443d9412fb49b02ce1437b553
W20260928 14:40:41.224753 5010 leader_election.cc:341] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22a55c393f224764b560f65f0526d943 (127.22.13.67:38323): Not found: Tablet not found: a5fcdc4443d9412fb49b02ce1437b553
I20260928 14:40:41.224824 5010 leader_election.cc:302] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ec98f35ee1546dea30d1610662f24d7; no voters: 22a55c393f224764b560f65f0526d943, f285265e654e42bfa1bc49224de0ac2d
I20260928 14:40:41.224946 5418 raft_consensus.cc:2726] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 14:40:41.342844 5419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:41.342631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:45855)
0928 14:40:41.342767 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:41.342796 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
I20260928 14:40:41.413921 5420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:41.413645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:52185)
0928 14:40:41.413825 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:41.413872 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":68,"threads_started":1}
I20260928 14:40:41.541046 5421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:41.540803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:53075)
0928 14:40:41.540954 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:41.540999 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260928 14:40:42.343559 5422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:42.343340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:51063)
0928 14:40:42.343471 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:42.343513 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260928 14:40:42.414844 5423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:42.414540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:46033)
0928 14:40:42.414711 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:42.414779 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":110,"threads_started":1}
I20260928 14:40:42.542013 5424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:42.541701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:42637)
0928 14:40:42.541858 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:42.541904 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":72,"threads_started":1}
I20260928 14:40:43.344292 5425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:43.344074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:55165)
0928 14:40:43.344207 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:43.344245 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260928 14:40:43.415639 5426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:43.415390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:35643)
0928 14:40:43.415541 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:43.415585 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":75,"threads_started":1}
I20260928 14:40:43.542822 5427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:43.542582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:32865)
0928 14:40:43.542728 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:43.542775 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260928 14:40:44.345063 5428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:44.344832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:40143)
0928 14:40:44.344978 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:44.345015 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260928 14:40:44.416358 5429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:44.416127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:37501)
0928 14:40:44.416267 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:44.416312 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":63,"threads_started":1}
I20260928 14:40:44.543673 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:44.543316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:39623)
0928 14:40:44.543566 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:44.543614 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":65,"threads_started":1}
I20260928 14:40:45.345892 5431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:45.345627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:49389)
0928 14:40:45.345797 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:45.345843 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260928 14:40:45.417330 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:45.416988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:40571)
0928 14:40:45.417233 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:45.417279 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":81,"threads_started":1}
I20260928 14:40:45.544610 5433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:45.544353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:49939)
0928 14:40:45.544516 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:45.544563 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":63,"threads_started":1}
I20260928 14:40:45.811303 5434 raft_consensus.cc:482] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:40:45.811422 5434 raft_consensus.cc:504] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f285265e654e42bfa1bc49224de0ac2d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 33767 } } peers { permanent_uuid: "22a55c393f224764b560f65f0526d943" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 38323 } } peers { permanent_uuid: "0ec98f35ee1546dea30d1610662f24d7" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 43987 } }
I20260928 14:40:45.811638 5434 leader_election.cc:288] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767), 22a55c393f224764b560f65f0526d943 (127.22.13.67:38323)
W20260928 14:40:45.812003 5010 leader_election.cc:341] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 22a55c393f224764b560f65f0526d943 (127.22.13.67:38323): Not found: Tablet not found: a5fcdc4443d9412fb49b02ce1437b553
W20260928 14:40:45.812067 5011 leader_election.cc:341] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f285265e654e42bfa1bc49224de0ac2d (127.22.13.66:33767): Not found: Tablet not found: a5fcdc4443d9412fb49b02ce1437b553
I20260928 14:40:45.812099 5011 leader_election.cc:302] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ec98f35ee1546dea30d1610662f24d7; no voters: 22a55c393f224764b560f65f0526d943, f285265e654e42bfa1bc49224de0ac2d
I20260928 14:40:45.812176 5434 raft_consensus.cc:2726] T a5fcdc4443d9412fb49b02ce1437b553 P 0ec98f35ee1546dea30d1610662f24d7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 14:40:46.346642 5435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:46.346435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:36463)
0928 14:40:46.346555 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:46.346589 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260928 14:40:46.418255 5436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:46.418025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:51315)
0928 14:40:46.418160 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:46.418207 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":60,"threads_started":1}
I20260928 14:40:46.545462 5437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:46.545247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:50227)
0928 14:40:46.545381 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:46.545414 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":78,"threads_started":1}
I20260928 14:40:47.347412 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:47.347179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:46221)
0928 14:40:47.347323 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:47.347362 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260928 14:40:47.418967 5439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:47.418748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:38725)
0928 14:40:47.418879 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:47.418917 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260928 14:40:47.546324 5440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:47.546096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:41763)
0928 14:40:47.546233 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:47.546275 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260928 14:40:48.348122 5441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:48.347895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:43121)
0928 14:40:48.348039 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:48.348073 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260928 14:40:48.419597 5442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:48.419401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:35587)
0928 14:40:48.419516 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:48.419549 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260928 14:40:48.547083 5443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:48.546871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:50497)
0928 14:40:48.546996 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:48.547034 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260928 14:40:49.348875 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:49.348653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.65:44801)
0928 14:40:49.348789 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:49.348829 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
I20260928 14:40:49.420392 5445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:49.420164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.66:60581)
0928 14:40:49.420302 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:49.420343 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260928 14:40:49.544866 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 4965
I20260928 14:40:49.547888 5446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:49.547681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41033 (local address 127.22.13.67:38741)
0928 14:40:49.547811 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:49.547844 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
I20260928 14:40:49.550302 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 5128
I20260928 14:40:49.554739 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 5261
2026-09-28T14:40:49Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18179 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:40:49 dist-test-slave-wwwb krb5kdc[5449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:40:49 dist-test-slave-wwwb krb5kdc[5449](info): set up 2 sockets
Sep 28 14:40:49 dist-test-slave-wwwb krb5kdc[5449](info): commencing operation
krb5kdc: starting...
W20260928 14:40:50.586345 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:40:50 dist-test-slave-wwwb krb5kdc[5449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:40:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:40:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:40:50.708292 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41223
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:41223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:50.792024 5462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:50.792279 5462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:50.792303 5462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:50.793506 5462 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 14:40:50.793548 5462 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 14:40:50.793561 5462 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 14:40:50.793573 5462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:40:50.793586 5462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:50.793601 5462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:50.793612 5462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:40:50.793624 5462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:40:50.795186 5462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:41223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:50.795416 5462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:50.795595 5462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:50.800738 5467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:50.800763 5468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:50.800782 5470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:50.800799 5462 server_base.cc:1034] running on GCE node
I20260928 14:40:50.801271 5462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:50.801659 5462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:50.802834 5462 hybrid_clock.cc:648] HybridClock initialized: now 1790606450802812 us; error 40 us; skew 500 ppm
Sep 28 14:40:50 dist-test-slave-wwwb krb5kdc[5449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606450, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:50.805457 5462 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:40:50.805979 5462 webserver.cc:460] Webserver started at http://127.22.13.126:42451/ using document root <none> and password file <none>
I20260928 14:40:50.806245 5462 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:50.806293 5462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:50.806447 5462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:40:50.807560 5462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "cefeac210b6047d38756f5f96a43f3e8"
format_stamp: "Formatted at 2026-09-28 14:40:50 on dist-test-slave-wwwb"
server_key: "0e2f7b6824feffc425bd809aa1929e2f"
server_key_iv: "08adbafdd7a9c704f63850c1f1c1bd70"
server_key_version: "encryptionkey@0"
I20260928 14:40:50.807941 5462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "cefeac210b6047d38756f5f96a43f3e8"
format_stamp: "Formatted at 2026-09-28 14:40:50 on dist-test-slave-wwwb"
server_key: "0e2f7b6824feffc425bd809aa1929e2f"
server_key_iv: "08adbafdd7a9c704f63850c1f1c1bd70"
server_key_version: "encryptionkey@0"
I20260928 14:40:50.809314 5462 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:40:50.810114 5477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:50.810290 5462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:50.810396 5462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "cefeac210b6047d38756f5f96a43f3e8"
format_stamp: "Formatted at 2026-09-28 14:40:50 on dist-test-slave-wwwb"
server_key: "0e2f7b6824feffc425bd809aa1929e2f"
server_key_iv: "08adbafdd7a9c704f63850c1f1c1bd70"
server_key_version: "encryptionkey@0"
I20260928 14:40:50.810500 5462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:50.823372 5462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:50.825479 5462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:50.825693 5462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:50.830322 5462 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:41223
I20260928 14:40:50.830369 5529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:41223 every 8 connection(s)
I20260928 14:40:50.830827 5462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:40:50.831773 5530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:50.833830 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 5462
I20260928 14:40:50.833994 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:40:50.834223 22581 external_mini_cluster.cc:1428] Setting key 240551420ed4d5ee0f97aab08bb8b405
I20260928 14:40:50.834488 5530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Bootstrap starting.
I20260928 14:40:50.835255 5530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:50.835580 5530 log.cc:824] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:50.836387 5530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: No bootstrap required, opened a new log
I20260928 14:40:50.838896 5530 raft_consensus.cc:348] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } }
I20260928 14:40:50.839145 5530 raft_consensus.cc:374] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:50.839180 5530 raft_consensus.cc:729] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cefeac210b6047d38756f5f96a43f3e8, State: Initialized, Role: FOLLOWER
I20260928 14:40:50.839311 5530 consensus_queue.cc:260] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } }
I20260928 14:40:50.839383 5530 raft_consensus.cc:388] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:40:50.839414 5530 raft_consensus.cc:482] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:40:50.839461 5530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:50.840195 5530 raft_consensus.cc:504] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } }
I20260928 14:40:50.840334 5530 leader_election.cc:302] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cefeac210b6047d38756f5f96a43f3e8; no voters:
I20260928 14:40:50.840545 5530 leader_election.cc:288] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:40:50.840613 5535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:40:50.840762 5535 raft_consensus.cc:686] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 1 LEADER]: Becoming Leader. State: Replica: cefeac210b6047d38756f5f96a43f3e8, State: Running, Role: LEADER
I20260928 14:40:50.840950 5530 sys_catalog.cc:564] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [sys.catalog]: configured and running, proceeding with master startup.
Sep 28 14:40:50 dist-test-slave-wwwb krb5kdc[5449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:50.842198 5535 consensus_queue.cc:237] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } }
I20260928 14:40:50.842806 5536 sys_catalog.cc:455] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cefeac210b6047d38756f5f96a43f3e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } } }
I20260928 14:40:50.842943 5536 sys_catalog.cc:458] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:50.842911 5535 sys_catalog.cc:455] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cefeac210b6047d38756f5f96a43f3e8. Latest consensus state: current_term: 1 leader_uuid: "cefeac210b6047d38756f5f96a43f3e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } } }
I20260928 14:40:50.843278 5535 sys_catalog.cc:458] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [sys.catalog]: This master's current role is: LEADER
I20260928 14:40:50.843739 5550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:40:50.844278 5550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:40:50.846096 5550 catalog_manager.cc:1347] Generated new cluster ID: 0ec02827e4f04bc5af6a11056249af72
I20260928 14:40:50.846158 5550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:40:50.847380 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:50.834999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48420 (local address 127.22.13.126:41223)
0928 14:40:50.835193 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:50.835201 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:50.835227 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:40:50.835764 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:50.835771 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:50.835823 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:50.835953 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:50.836711 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:50.838177 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:50.838979 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:50.839623 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:50.839748 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:50.842883 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:50.842887 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:50.842888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:50.842902 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:50.845211 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:50.845712 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:50.845715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:50.845715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:50.845829 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:50.846085 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:50.846086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:50.846086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:50.846420 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:50.846553 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:50.847004 (+ 451us) server_negotiation.cc:299] Negotiation successful
0928 14:40:50.847153 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
I20260928 14:40:50.854348 5550 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:40:50.855170 5550 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:40:50.857766 5550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Generated new TSK 0
I20260928 14:40:50.858016 5550 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:40:50.886595 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:41223
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:50.972954 5556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:50.973193 5556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:50.973223 5556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:50.974558 5556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:50.974610 5556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:50.974640 5556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:40:50.976282 5556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:41223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:50.976569 5556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:50.976825 5556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:50.982219 5564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:50.982391 5556 server_base.cc:1034] running on GCE node
W20260928 14:40:50.982214 5562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:50.982234 5561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:50.982821 5556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:50.983059 5556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:50.984213 5556 hybrid_clock.cc:648] HybridClock initialized: now 1790606450984201 us; error 30 us; skew 500 ppm
Sep 28 14:40:50 dist-test-slave-wwwb krb5kdc[5449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606450, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:50.986871 5556 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:40:50.987283 5556 webserver.cc:460] Webserver started at http://127.22.13.65:38157/ using document root <none> and password file <none>
I20260928 14:40:50.987475 5556 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:50.987516 5556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:50.987619 5556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:40:50.988685 5556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "22e98f4d9dd9467c92809fdd2c8eaf79"
format_stamp: "Formatted at 2026-09-28 14:40:50 on dist-test-slave-wwwb"
server_key: "788f8365bf2825945ac3fdd2498901c1"
server_key_iv: "9d7a5e65dcb62dce5aab49e6389d1dd5"
server_key_version: "encryptionkey@0"
I20260928 14:40:50.988994 5556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "22e98f4d9dd9467c92809fdd2c8eaf79"
format_stamp: "Formatted at 2026-09-28 14:40:50 on dist-test-slave-wwwb"
server_key: "788f8365bf2825945ac3fdd2498901c1"
server_key_iv: "9d7a5e65dcb62dce5aab49e6389d1dd5"
server_key_version: "encryptionkey@0"
I20260928 14:40:50.990377 5556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:40:50.991129 5571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:50.991307 5556 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:40:50.991410 5556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "22e98f4d9dd9467c92809fdd2c8eaf79"
format_stamp: "Formatted at 2026-09-28 14:40:50 on dist-test-slave-wwwb"
server_key: "788f8365bf2825945ac3fdd2498901c1"
server_key_iv: "9d7a5e65dcb62dce5aab49e6389d1dd5"
server_key_version: "encryptionkey@0"
I20260928 14:40:50.991501 5556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:51.009456 5556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:51.011511 5556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:51.011700 5556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:51.011965 5556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:51.012346 5556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:40:51.012380 5556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:51.012403 5556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:40:51.012416 5556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:51.018290 5556 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:38419
I20260928 14:40:51.018882 5556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:40:51.018337 5684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:38419 every 8 connection(s)
I20260928 14:40:51.022327 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 5556
I20260928 14:40:51.022464 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:40:51.022681 22581 external_mini_cluster.cc:1428] Setting key 52a5a94f95020fbe70e9d7f863a32beb
Sep 28 14:40:51 dist-test-slave-wwwb krb5kdc[5449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606450, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:51.030268 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.019791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:50661 (local address 127.22.13.126:41223)
0928 14:40:51.019864 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:51.019865 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:51.020049 (+ 184us) server_negotiation.cc:407] Connection header received
0928 14:40:51.020575 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:51.020576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:51.020587 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:51.020653 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:51.021646 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.022301 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.023656 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.023992 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.024073 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.026776 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:51.026778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:51.026779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:51.026788 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:51.028658 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:51.029126 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:51.029128 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:51.029129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:51.029202 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:51.029451 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:51.029452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:51.029453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:51.029640 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:51.029722 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:51.030017 (+ 295us) server_negotiation.cc:299] Negotiation successful
0928 14:40:51.030132 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 14:40:51.030422 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.019761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:50661)
0928 14:40:51.019926 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:51.019972 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:51.020482 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:51.020721 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:51.020725 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:51.021331 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:51.021546 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:51.021552 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.022427 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.022428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:51.023564 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:51.023567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.024088 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.024089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:51.024207 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.024849 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:51.024863 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:51.026660 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:51.028773 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:51.028782 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:51.028784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:51.029043 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:51.029277 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:51.029277 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:51.029278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:51.029384 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:51.029711 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:51.029716 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:51.029848 (+ 132us) client_negotiation.cc:769] Sending connection context
0928 14:40:51.029994 (+ 146us) client_negotiation.cc:240] Negotiation successful
0928 14:40:51.030154 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260928 14:40:51.031014 5685 heartbeater.cc:347] Connected to a master server at 127.22.13.126:41223
I20260928 14:40:51.031164 5685 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:51.031376 5685 heartbeater.cc:511] Master 127.22.13.126:41223 requested a full tablet report, sending...
I20260928 14:40:51.032058 5494 ts_manager.cc:194] Registered new tserver with Master: 22e98f4d9dd9467c92809fdd2c8eaf79 (127.22.13.65:38419)
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
I20260928 14:40:51.032994 5494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:50661
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:40:51.050501 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:41223
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:51.133296 5690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:51.133558 5690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:51.133596 5690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:51.134801 5690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:51.134877 5690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:51.134923 5690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:40:51.136415 5690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:41223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:51.136706 5690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:51.136950 5690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:40:51.142076 5696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:51.142076 5698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:51.142118 5690 server_base.cc:1034] running on GCE node
W20260928 14:40:51.142079 5695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:51.142634 5690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:51.142951 5690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:51.144182 5690 hybrid_clock.cc:648] HybridClock initialized: now 1790606451144157 us; error 40 us; skew 500 ppm
Sep 28 14:40:51 dist-test-slave-wwwb krb5kdc[5449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606451, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:51.146987 5690 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:40:51.147464 5690 webserver.cc:460] Webserver started at http://127.22.13.66:37605/ using document root <none> and password file <none>
I20260928 14:40:51.147701 5690 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:51.147747 5690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:51.147924 5690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:40:51.149034 5690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "ee9c8b5cc3ac444ab1551d1634c47e82"
format_stamp: "Formatted at 2026-09-28 14:40:51 on dist-test-slave-wwwb"
server_key: "dbc961026bc0aeac41e9429e2a575cc8"
server_key_iv: "6ecadece95371e6d67bfb086695bc318"
server_key_version: "encryptionkey@0"
I20260928 14:40:51.149420 5690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "ee9c8b5cc3ac444ab1551d1634c47e82"
format_stamp: "Formatted at 2026-09-28 14:40:51 on dist-test-slave-wwwb"
server_key: "dbc961026bc0aeac41e9429e2a575cc8"
server_key_iv: "6ecadece95371e6d67bfb086695bc318"
server_key_version: "encryptionkey@0"
I20260928 14:40:51.150945 5690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:40:51.151791 5705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:51.151986 5690 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:51.152082 5690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "ee9c8b5cc3ac444ab1551d1634c47e82"
format_stamp: "Formatted at 2026-09-28 14:40:51 on dist-test-slave-wwwb"
server_key: "dbc961026bc0aeac41e9429e2a575cc8"
server_key_iv: "6ecadece95371e6d67bfb086695bc318"
server_key_version: "encryptionkey@0"
I20260928 14:40:51.152163 5690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:51.164283 5690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:51.166520 5690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:51.166683 5690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:51.166951 5690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:51.167286 5690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:40:51.167317 5690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:51.167340 5690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:40:51.167352 5690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:51.173794 5690 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:41107
I20260928 14:40:51.173866 5818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:41107 every 8 connection(s)
I20260928 14:40:51.174201 5690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:40:51.175817 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 5690
I20260928 14:40:51.175941 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:40:51.176168 22581 external_mini_cluster.cc:1428] Setting key f1e34b2841ea84866bc368b4007d76e2
Sep 28 14:40:51 dist-test-slave-wwwb krb5kdc[5449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606451, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
I20260928 14:40:51.185170 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.174999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:40957 (local address 127.22.13.126:41223)
0928 14:40:51.175082 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:51.175085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:51.175328 (+ 243us) server_negotiation.cc:407] Connection header received
0928 14:40:51.176005 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:51.176007 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:51.176021 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:51.176054 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:51.177362 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.177876 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.179118 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.179337 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.179423 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.182230 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:51.182232 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:51.182232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:51.182240 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:51.183677 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:51.184203 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:51.184204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:51.184204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:51.184258 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:51.184568 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:51.184569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:51.184570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:51.184689 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:51.184745 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:51.185025 (+ 280us) server_negotiation.cc:299] Negotiation successful
0928 14:40:51.185099 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 14:40:51.185351 5821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.174988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:40957)
0928 14:40:51.175196 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:51.175233 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:51.175852 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:51.176268 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:51.176272 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:51.176912 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:51.177242 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:51.177249 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.178024 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.178026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:51.178997 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:51.179000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.179436 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.179437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:51.179571 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.180177 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:51.180190 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:51.182122 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:51.183780 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:51.183791 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:51.183794 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:51.184118 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:51.184330 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:51.184332 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:51.184332 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:51.184498 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:51.184766 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:51.184771 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:51.184905 (+ 134us) client_negotiation.cc:769] Sending connection context
0928 14:40:51.184997 (+ 92us) client_negotiation.cc:240] Negotiation successful
0928 14:40:51.185132 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":117,"threads_started":1}
I20260928 14:40:51.185907 5819 heartbeater.cc:347] Connected to a master server at 127.22.13.126:41223
I20260928 14:40:51.186055 5819 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:51.186224 5819 heartbeater.cc:511] Master 127.22.13.126:41223 requested a full tablet report, sending...
I20260928 14:40:51.186640 5494 ts_manager.cc:194] Registered new tserver with Master: ee9c8b5cc3ac444ab1551d1634c47e82 (127.22.13.66:41107)
I20260928 14:40:51.187033 5494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:40957
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:40:51.202160 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:41223
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:40:51.283874 5824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:40:51.284142 5824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:40:51.284186 5824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:40:51.285413 5824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:40:51.285495 5824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:40:51.285542 5824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:40:51.287108 5824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:41223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:40:51.287395 5824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:40:51.287638 5824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:40:51.292930 5824 server_base.cc:1034] running on GCE node
W20260928 14:40:51.292835 5829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:51.292882 5830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:40:51.292834 5832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:40:51.293323 5824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:40:51.293627 5824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:40:51.294790 5824 hybrid_clock.cc:648] HybridClock initialized: now 1790606451294776 us; error 29 us; skew 500 ppm
Sep 28 14:40:51 dist-test-slave-wwwb krb5kdc[5449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606451, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:40:51.297391 5824 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:40:51.297897 5824 webserver.cc:460] Webserver started at http://127.22.13.67:33409/ using document root <none> and password file <none>
I20260928 14:40:51.298156 5824 fs_manager.cc:359] Metadata directory not provided
I20260928 14:40:51.298228 5824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:40:51.298389 5824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:40:51.299470 5824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "a0ed8637b495448fbf2edae1c8aa4161"
format_stamp: "Formatted at 2026-09-28 14:40:51 on dist-test-slave-wwwb"
server_key: "23972ce1adb883570ab5bd70f174a4bf"
server_key_iv: "bd1ce638133273522b782978b551c787"
server_key_version: "encryptionkey@0"
I20260928 14:40:51.299851 5824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "a0ed8637b495448fbf2edae1c8aa4161"
format_stamp: "Formatted at 2026-09-28 14:40:51 on dist-test-slave-wwwb"
server_key: "23972ce1adb883570ab5bd70f174a4bf"
server_key_iv: "bd1ce638133273522b782978b551c787"
server_key_version: "encryptionkey@0"
I20260928 14:40:51.301213 5824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:40:51.302201 5839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:51.302373 5824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:40:51.302490 5824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "a0ed8637b495448fbf2edae1c8aa4161"
format_stamp: "Formatted at 2026-09-28 14:40:51 on dist-test-slave-wwwb"
server_key: "23972ce1adb883570ab5bd70f174a4bf"
server_key_iv: "bd1ce638133273522b782978b551c787"
server_key_version: "encryptionkey@0"
I20260928 14:40:51.302569 5824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:40:51.314321 5824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:40:51.316193 5824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:40:51.316368 5824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:40:51.316622 5824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:40:51.316946 5824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:40:51.317000 5824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:51.317047 5824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:40:51.317080 5824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:40:51.323100 5824 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:37733
I20260928 14:40:51.323151 5952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:37733 every 8 connection(s)
I20260928 14:40:51.323467 5824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:40:51.327243 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 5824
I20260928 14:40:51.327370 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:40:51.327574 22581 external_mini_cluster.cc:1428] Setting key 09bd06cb8792a97d209f975adb5e8e95
Sep 28 14:40:51 dist-test-slave-wwwb krb5kdc[5449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606451, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:40:51.332840 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.324286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:55171 (local address 127.22.13.126:41223)
0928 14:40:51.324378 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:51.324380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:51.324549 (+ 169us) server_negotiation.cc:407] Connection header received
0928 14:40:51.325017 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:51.325018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:51.325029 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:51.325082 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:51.326114 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.326617 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.327703 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.327918 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.327979 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.330289 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:51.330290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:51.330290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:51.330297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:51.331710 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:51.332060 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:51.332061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:51.332061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:51.332111 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:51.332320 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:51.332321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:51.332321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:51.332424 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:51.332478 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:51.332709 (+ 231us) server_negotiation.cc:299] Negotiation successful
0928 14:40:51.332774 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 14:40:51.333056 5955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.324236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:55171)
0928 14:40:51.324411 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:51.324450 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:51.324930 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:51.325141 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:51.325145 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:51.325746 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:40:51.326023 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:51.326029 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.326703 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.326704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:51.327606 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:51.327609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.327993 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.327994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:51.328150 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.328770 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:40:51.328779 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:40:51.330206 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:40:51.331786 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:51.331792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:51.331794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:51.331998 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:51.332168 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:40:51.332168 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:40:51.332169 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:40:51.332262 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:40:51.332477 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:40:51.332481 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:40:51.332595 (+ 114us) client_negotiation.cc:769] Sending connection context
0928 14:40:51.332684 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 14:40:51.332805 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":106,"threads_started":1}
I20260928 14:40:51.333582 5953 heartbeater.cc:347] Connected to a master server at 127.22.13.126:41223
I20260928 14:40:51.333681 5953 heartbeater.cc:464] Registering TS with master...
I20260928 14:40:51.333827 5953 heartbeater.cc:511] Master 127.22.13.126:41223 requested a full tablet report, sending...
I20260928 14:40:51.334237 5494 ts_manager.cc:194] Registered new tserver with Master: a0ed8637b495448fbf2edae1c8aa4161 (127.22.13.67:37733)
I20260928 14:40:51.334631 5494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:55171
I20260928 14:40:51.339382 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:40:51.347618 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.340849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48428 (local address 127.22.13.126:41223)
0928 14:40:51.340908 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:51.340910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:51.341035 (+ 125us) server_negotiation.cc:407] Connection header received
0928 14:40:51.341159 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:51.341160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:51.341170 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:51.341242 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:40:51.341875 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.342494 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.343286 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.343569 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.343648 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.344942 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:40:51.344950 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:40:51.344950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:40:51.344957 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:40:51.346404 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:51.346786 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:51.346786 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:51.346787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:51.346847 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:40:51.347053 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:40:51.347054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:40:51.347054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:40:51.347207 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:40:51.347315 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:51.347419 (+ 104us) server_negotiation.cc:299] Negotiation successful
0928 14:40:51.347509 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 14:40:51.349350 5494 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:40:51.350991 5494 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:40:51.357657 5963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.353305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46748 (local address 127.22.13.65:38419)
0928 14:40:51.353761 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:51.353767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:51.353777 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:40:51.353818 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:51.353821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:51.353836 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:51.353909 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:51.354172 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.355069 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.356694 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.357204 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.357302 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.357367 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:51.357526 (+ 159us) server_negotiation.cc:299] Negotiation successful
0928 14:40:51.357568 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":80,"threads_started":1}
I20260928 14:40:51.357658 5964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.353172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38419 (local address 127.0.0.1:46748)
0928 14:40:51.353554 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:51.353565 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:51.353633 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:51.353904 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:51.353905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:51.353907 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:51.354009 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:51.354011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.355170 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.355171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:51.356606 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:51.356607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.357304 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.357305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:51.357386 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.357401 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 14:40:51.357505 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 14:40:51.357567 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":56,"thread_start_us":58,"threads_started":1}
I20260928 14:40:51.357766 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.353376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:37733 (local address 127.0.0.1:35386)
0928 14:40:51.353718 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:51.353728 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:51.353773 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:51.354046 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:51.354047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:51.354049 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:51.354085 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:51.354086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.355383 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.355384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:51.356502 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:51.356505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.357521 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.357522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:51.357577 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.357581 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:40:51.357633 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 14:40:51.357692 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":2,"thread_start_us":36,"threads_started":1}
I20260928 14:40:51.357790 5965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.353375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36308 (local address 127.22.13.66:41107)
0928 14:40:51.353554 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:51.353559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:51.353569 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:40:51.353614 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:51.353616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:51.353647 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:51.353728 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:51.354040 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.355256 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.356657 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.357352 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.357412 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.357467 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:51.357679 (+ 212us) server_negotiation.cc:299] Negotiation successful
0928 14:40:51.357719 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":91,"threads_started":1}
I20260928 14:40:51.358021 5961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.353162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:41107 (local address 127.0.0.1:36308)
0928 14:40:51.353343 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:51.353387 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:51.353453 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:51.353830 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:51.353834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:51.353837 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:51.353919 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:51.353922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.355366 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.355367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:51.356570 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:51.356572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.357506 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.357507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:51.357581 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.357583 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:40:51.357633 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 14:40:51.357915 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":275,"thread_start_us":80,"threads_started":1}
I20260928 14:40:51.358023 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.353537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35386 (local address 127.22.13.67:37733)
0928 14:40:51.353748 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:51.353752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:51.353764 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:40:51.353842 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:51.353844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:51.353858 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:51.354083 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:51.354168 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.355274 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.356588 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.357335 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.357412 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.357467 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:51.357870 (+ 403us) server_negotiation.cc:299] Negotiation successful
0928 14:40:51.357915 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":94,"threads_started":1}
I20260928 14:40:51.358861 5753 tablet_service.cc:1467] Processing CreateTablet for tablet b196aa0b46364c2f9801926c8574e3e1 (DEFAULT_TABLE table=test-table [id=a7cb9058aab048b8bfcb1face5aec0ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:40:51.359037 5887 tablet_service.cc:1467] Processing CreateTablet for tablet b196aa0b46364c2f9801926c8574e3e1 (DEFAULT_TABLE table=test-table [id=a7cb9058aab048b8bfcb1face5aec0ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:40:51.359300 5753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b196aa0b46364c2f9801926c8574e3e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:51.359230 5619 tablet_service.cc:1467] Processing CreateTablet for tablet b196aa0b46364c2f9801926c8574e3e1 (DEFAULT_TABLE table=test-table [id=a7cb9058aab048b8bfcb1face5aec0ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:40:51.359388 5887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b196aa0b46364c2f9801926c8574e3e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:51.359519 5619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b196aa0b46364c2f9801926c8574e3e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:40:51.362011 5967 tablet_bootstrap.cc:492] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82: Bootstrap starting.
I20260928 14:40:51.362712 5967 tablet_bootstrap.cc:654] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:51.362829 5969 tablet_bootstrap.cc:492] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: Bootstrap starting.
I20260928 14:40:51.363525 5969 tablet_bootstrap.cc:654] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:51.363538 5967 log.cc:824] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:51.364115 5969 log.cc:824] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:51.364331 5967 tablet_bootstrap.cc:492] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82: No bootstrap required, opened a new log
I20260928 14:40:51.364406 5967 ts_tablet_manager.cc:1397] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:40:51.364470 5968 tablet_bootstrap.cc:492] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161: Bootstrap starting.
I20260928 14:40:51.365113 5968 tablet_bootstrap.cc:654] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161: Neither blocks nor log segments found. Creating new log.
I20260928 14:40:51.365674 5968 log.cc:824] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161: Log is configured to *not* fsync() on all Append() calls
I20260928 14:40:51.366281 5969 tablet_bootstrap.cc:492] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: No bootstrap required, opened a new log
I20260928 14:40:51.366411 5969 ts_tablet_manager.cc:1397] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260928 14:40:51.366556 5968 tablet_bootstrap.cc:492] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161: No bootstrap required, opened a new log
I20260928 14:40:51.366698 5968 ts_tablet_manager.cc:1397] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:40:51.366772 5967 raft_consensus.cc:348] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } } peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } }
I20260928 14:40:51.366964 5967 raft_consensus.cc:374] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:51.366997 5967 raft_consensus.cc:729] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9c8b5cc3ac444ab1551d1634c47e82, State: Initialized, Role: FOLLOWER
I20260928 14:40:51.367153 5967 consensus_queue.cc:260] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } } peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } }
I20260928 14:40:51.367432 5967 ts_tablet_manager.cc:1428] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:40:51.367579 5819 heartbeater.cc:503] Master 127.22.13.126:41223 was elected leader, sending a full tablet report...
I20260928 14:40:51.368242 5969 raft_consensus.cc:348] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } } peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } }
I20260928 14:40:51.368367 5969 raft_consensus.cc:374] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:51.368387 5969 raft_consensus.cc:729] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22e98f4d9dd9467c92809fdd2c8eaf79, State: Initialized, Role: FOLLOWER
I20260928 14:40:51.368512 5969 consensus_queue.cc:260] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } } peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } }
I20260928 14:40:51.368803 5685 heartbeater.cc:503] Master 127.22.13.126:41223 was elected leader, sending a full tablet report...
I20260928 14:40:51.368801 5969 ts_tablet_manager.cc:1428] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 14:40:51.368817 5968 raft_consensus.cc:348] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } } peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } }
I20260928 14:40:51.368975 5968 raft_consensus.cc:374] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:40:51.368999 5968 raft_consensus.cc:729] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ed8637b495448fbf2edae1c8aa4161, State: Initialized, Role: FOLLOWER
I20260928 14:40:51.369127 5968 consensus_queue.cc:260] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } } peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } }
I20260928 14:40:51.369400 5968 ts_tablet_manager.cc:1428] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:40:51.369426 5953 heartbeater.cc:503] Master 127.22.13.126:41223 was elected leader, sending a full tablet report...
W20260928 14:40:51.424817 5820 tablet.cc:2369] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:40:51.519965 5686 tablet.cc:2369] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:40:51.574220 5954 tablet.cc:2369] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:40:51.596637 5974 raft_consensus.cc:482] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:40:51.596805 5974 raft_consensus.cc:504] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } } peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } }
I20260928 14:40:51.597258 5974 leader_election.cc:288] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee9c8b5cc3ac444ab1551d1634c47e82 (127.22.13.66:41107), a0ed8637b495448fbf2edae1c8aa4161 (127.22.13.67:37733)
I20260928 14:40:51.600299 5965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.597661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:35883 (local address 127.22.13.66:41107)
0928 14:40:51.597714 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:51.597715 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:51.597727 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:40:51.597782 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:51.597783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:51.597794 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:51.597840 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:51.598035 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.598516 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.599562 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.599940 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.600028 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.600037 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:51.600186 (+ 149us) server_negotiation.cc:299] Negotiation successful
0928 14:40:51.600225 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 14:40:51.600317 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.597435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:41107 (local address 127.22.13.65:35883)
0928 14:40:51.597639 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:51.597650 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:51.597715 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:51.597852 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:51.597853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:51.597855 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:51.597956 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:51.597958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.598618 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.598620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:51.599463 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:51.599465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.600028 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.600029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:51.600099 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.600103 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:40:51.600181 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:40:51.600226 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":33,"thread_start_us":68,"threads_started":1}
I20260928 14:40:51.600380 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.597590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:48801 (local address 127.22.13.67:37733)
0928 14:40:51.597641 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:51.597644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:51.597804 (+ 160us) server_negotiation.cc:407] Connection header received
0928 14:40:51.597899 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:51.597900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:51.597909 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:51.598037 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:40:51.598209 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.598668 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.599631 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.600013 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.600111 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.600119 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:51.600272 (+ 153us) server_negotiation.cc:299] Negotiation successful
0928 14:40:51.600311 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:40:51.600423 5977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.597510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:37733 (local address 127.22.13.65:48801)
0928 14:40:51.597704 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:51.597716 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:40:51.597819 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:40:51.598044 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:40:51.598045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:40:51.598048 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:40:51.598127 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:51.598129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.598751 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.598753 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:51.599534 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:40:51.599536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.600111 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.600111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:40:51.600187 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.600190 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:40:51.600263 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 14:40:51.600304 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260928 14:40:51.600770 5773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b196aa0b46364c2f9801926c8574e3e1" candidate_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" is_pre_election: true
I20260928 14:40:51.600890 5907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b196aa0b46364c2f9801926c8574e3e1" candidate_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ed8637b495448fbf2edae1c8aa4161" is_pre_election: true
I20260928 14:40:51.600944 5773 raft_consensus.cc:2454] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22e98f4d9dd9467c92809fdd2c8eaf79 in term 0.
I20260928 14:40:51.601020 5907 raft_consensus.cc:2454] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22e98f4d9dd9467c92809fdd2c8eaf79 in term 0.
I20260928 14:40:51.601200 5573 leader_election.cc:302] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22e98f4d9dd9467c92809fdd2c8eaf79, ee9c8b5cc3ac444ab1551d1634c47e82; no voters:
I20260928 14:40:51.601356 5974 raft_consensus.cc:2781] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:40:51.601413 5974 raft_consensus.cc:482] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:40:51.601447 5974 raft_consensus.cc:3037] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:51.602250 5974 raft_consensus.cc:504] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } } peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } }
I20260928 14:40:51.602406 5974 leader_election.cc:288] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [CANDIDATE]: Term 1 election: Requested vote from peers ee9c8b5cc3ac444ab1551d1634c47e82 (127.22.13.66:41107), a0ed8637b495448fbf2edae1c8aa4161 (127.22.13.67:37733)
I20260928 14:40:51.602576 5773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b196aa0b46364c2f9801926c8574e3e1" candidate_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82"
I20260928 14:40:51.602605 5907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b196aa0b46364c2f9801926c8574e3e1" candidate_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ed8637b495448fbf2edae1c8aa4161"
I20260928 14:40:51.602671 5773 raft_consensus.cc:3037] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:51.602691 5907 raft_consensus.cc:3037] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:40:51.603370 5907 raft_consensus.cc:2454] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22e98f4d9dd9467c92809fdd2c8eaf79 in term 1.
I20260928 14:40:51.603477 5773 raft_consensus.cc:2454] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22e98f4d9dd9467c92809fdd2c8eaf79 in term 1.
I20260928 14:40:51.603595 5572 leader_election.cc:302] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22e98f4d9dd9467c92809fdd2c8eaf79, a0ed8637b495448fbf2edae1c8aa4161; no voters:
I20260928 14:40:51.603704 5974 raft_consensus.cc:2781] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:40:51.603865 5974 raft_consensus.cc:686] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 1 LEADER]: Becoming Leader. State: Replica: 22e98f4d9dd9467c92809fdd2c8eaf79, State: Running, Role: LEADER
I20260928 14:40:51.603995 5974 consensus_queue.cc:237] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } } peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } }
I20260928 14:40:51.604866 5493 catalog_manager.cc:5495] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22e98f4d9dd9467c92809fdd2c8eaf79 (127.22.13.65). New cstate: current_term: 1 leader_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } health_report { overall_health: UNKNOWN } } }
I20260928 14:40:51.642620 5963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.639803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46758 (local address 127.22.13.65:38419)
0928 14:40:51.639854 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:40:51.639856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:40:51.639866 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:40:51.639916 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:40:51.639917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:40:51.639942 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:40:51.640023 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:40:51.640184 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.640804 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.641687 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:40:51.641990 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:40:51.642064 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:40:51.642257 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:40:51.642375 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:40:51.642488 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0928 14:40:51.642508 (+ 20us) server_negotiation.cc:299] Negotiation successful
0928 14:40:51.642530 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:40:51.644511 5907 raft_consensus.cc:1261] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161 [term 1 FOLLOWER]: Refusing update from remote peer 22e98f4d9dd9467c92809fdd2c8eaf79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:40:51.644541 5773 raft_consensus.cc:1261] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82 [term 1 FOLLOWER]: Refusing update from remote peer 22e98f4d9dd9467c92809fdd2c8eaf79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:40:51.644933 5978 consensus_queue.cc:1035] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:40:51.645092 5974 consensus_queue.cc:1035] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:40:51.647342 5981 mvcc.cc:205] Tried to move back new op lower bound from 7334324025933017088 to 7334324025770631168. Current Snapshot: MvccSnapshot[applied={T|T < 7334324025933017088}]
I20260928 14:40:51.647800 5982 mvcc.cc:205] Tried to move back new op lower bound from 7334324025933017088 to 7334324025770631168. Current Snapshot: MvccSnapshot[applied={T|T < 7334324025933017088}]
I20260928 14:40:51.647974 5983 mvcc.cc:205] Tried to move back new op lower bound from 7334324025933017088 to 7334324025770631168. Current Snapshot: MvccSnapshot[applied={T|T < 7334324025933017088}]
I20260928 14:40:51.650308 5494 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:40:51.650409 5494 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:40:51.651189 5494 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Sending DeleteTablet for 3 replicas of tablet b196aa0b46364c2f9801926c8574e3e1
I20260928 14:40:51.651507 5753 tablet_service.cc:1514] Processing DeleteTablet for tablet b196aa0b46364c2f9801926c8574e3e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:40:51 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:36308
I20260928 14:40:51.651573 5619 tablet_service.cc:1514] Processing DeleteTablet for tablet b196aa0b46364c2f9801926c8574e3e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:40:51 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:46748
I20260928 14:40:51.651589 5887 tablet_service.cc:1514] Processing DeleteTablet for tablet b196aa0b46364c2f9801926c8574e3e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:40:51 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:35386
I20260928 14:40:51.651800 5993 tablet_replica.cc:330] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: stopping tablet replica
I20260928 14:40:51.651875 5991 tablet_replica.cc:330] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82: stopping tablet replica
I20260928 14:40:51.651880 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 5556
I20260928 14:40:51.651898 5993 raft_consensus.cc:2229] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:40:51.651952 5991 raft_consensus.cc:2229] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:40:51.652016 5993 raft_consensus.cc:2258] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:40:51.652034 5991 raft_consensus.cc:2258] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:40:51.652428 5992 tablet_replica.cc:330] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161: stopping tablet replica
I20260928 14:40:51.652456 5991 ts_tablet_manager.cc:1905] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:40:51.652513 5992 raft_consensus.cc:2229] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:40:51.652599 5992 raft_consensus.cc:2258] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:40:51.653072 5992 ts_tablet_manager.cc:1905] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:40:51.654379 5991 ts_tablet_manager.cc:1918] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:40:51.654443 5991 log.cc:1196] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/b196aa0b46364c2f9801926c8574e3e1
I20260928 14:40:51.654762 5991 ts_tablet_manager.cc:1939] T b196aa0b46364c2f9801926c8574e3e1 P ee9c8b5cc3ac444ab1551d1634c47e82: Deleting consensus metadata
I20260928 14:40:51.654829 5992 ts_tablet_manager.cc:1918] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:40:51.654913 5992 log.cc:1196] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/b196aa0b46364c2f9801926c8574e3e1
I20260928 14:40:51.655238 5992 ts_tablet_manager.cc:1939] T b196aa0b46364c2f9801926c8574e3e1 P a0ed8637b495448fbf2edae1c8aa4161: Deleting consensus metadata
I20260928 14:40:51.655220 5479 catalog_manager.cc:4841] TS ee9c8b5cc3ac444ab1551d1634c47e82 (127.22.13.66:41107): tablet b196aa0b46364c2f9801926c8574e3e1 (table test-table [id=a7cb9058aab048b8bfcb1face5aec0ff]) successfully deleted
I20260928 14:40:51.655644 5478 catalog_manager.cc:4841] TS a0ed8637b495448fbf2edae1c8aa4161 (127.22.13.67:37733): tablet b196aa0b46364c2f9801926c8574e3e1 (table test-table [id=a7cb9058aab048b8bfcb1face5aec0ff]) successfully deleted
W20260928 14:40:51.658129 5478 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:38419 (error 108)
I20260928 14:40:51.658483 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 5690
I20260928 14:40:51.658658 5961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:40:51.658500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38419 (local address 127.0.0.1:46760)
0928 14:40:51.658560 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0928 14:40:51.658605 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:38419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 14:40:51.662101 5478 catalog_manager.cc:4584] TS 22e98f4d9dd9467c92809fdd2c8eaf79 (127.22.13.65:38419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b196aa0b46364c2f9801926c8574e3e1: Network error: Client connection negotiation failed: client connection to 127.22.13.65:38419: connect: Connection refused (error 111)
I20260928 14:40:51.663527 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 5824
I20260928 14:40:51.668335 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 5462
I20260928 14:41:11.673629 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41223
--webserver_interface=127.22.13.126
--webserver_port=42451
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:41223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:11.759678 5995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:11.760044 5995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:11.760087 5995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:11.761536 5995 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 14:41:11.761591 5995 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 14:41:11.761605 5995 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 14:41:11.761617 5995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:41:11.761631 5995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:11.761644 5995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:11.761704 5995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:41:11.761721 5995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:41:11.763486 5995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:41223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=42451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:11.763799 5995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:11.764086 5995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:11.773360 6001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:11.773336 6004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:11.773422 5995 server_base.cc:1034] running on GCE node
W20260928 14:41:11.773336 6002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:11.773797 5995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:11.774178 5995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:11.775353 5995 hybrid_clock.cc:648] HybridClock initialized: now 1790606471775348 us; error 38 us; skew 500 ppm
Sep 28 14:41:11 dist-test-slave-wwwb krb5kdc[5449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606471, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:11.782581 5995 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:41:11.783053 5995 webserver.cc:460] Webserver started at http://127.22.13.126:42451/ using document root <none> and password file <none>
I20260928 14:41:11.783277 5995 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:11.783332 5995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:11.784829 5995 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:11.785540 6011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:11.785768 5995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:11.785866 5995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "cefeac210b6047d38756f5f96a43f3e8"
format_stamp: "Formatted at 2026-09-28 14:40:50 on dist-test-slave-wwwb"
server_key: "0e2f7b6824feffc425bd809aa1929e2f"
server_key_iv: "08adbafdd7a9c704f63850c1f1c1bd70"
server_key_version: "encryptionkey@0"
I20260928 14:41:11.786366 5995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:11.798235 5995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:11.800745 5995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:11.800876 5995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:11.804976 5995 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:41223
I20260928 14:41:11.805053 6063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:41223 every 8 connection(s)
I20260928 14:41:11.805483 5995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:41:11.807230 6064 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:41:11.807948 6064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Bootstrap starting.
I20260928 14:41:11.809144 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 5995
I20260928 14:41:11.809463 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:38419
--local_ip_for_outbound_sockets=127.22.13.65
--tserver_master_addrs=127.22.13.126:41223
--webserver_port=38157
--webserver_interface=127.22.13.65
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:41:11.817219 6064 log.cc:824] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:11.819358 6064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:41:11.819715 6064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Bootstrap complete.
I20260928 14:41:11.822194 6064 raft_consensus.cc:348] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } }
I20260928 14:41:11.822485 6064 raft_consensus.cc:729] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cefeac210b6047d38756f5f96a43f3e8, State: Initialized, Role: FOLLOWER
I20260928 14:41:11.822679 6064 consensus_queue.cc:260] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } }
I20260928 14:41:11.822762 6064 raft_consensus.cc:388] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:41:11.822820 6064 raft_consensus.cc:482] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:41:11.822887 6064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:41:11.824060 6064 raft_consensus.cc:504] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } }
I20260928 14:41:11.824190 6064 leader_election.cc:302] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cefeac210b6047d38756f5f96a43f3e8; no voters:
I20260928 14:41:11.824410 6064 leader_election.cc:288] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 14:41:11.824452 6068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 14:41:11.824584 6068 raft_consensus.cc:686] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 2 LEADER]: Becoming Leader. State: Replica: cefeac210b6047d38756f5f96a43f3e8, State: Running, Role: LEADER
I20260928 14:41:11.824705 6064 sys_catalog.cc:564] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:41:11.824767 6068 consensus_queue.cc:237] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } }
I20260928 14:41:11.825155 6069 sys_catalog.cc:455] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cefeac210b6047d38756f5f96a43f3e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } } }
I20260928 14:41:11.825243 6069 sys_catalog.cc:458] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:11.825606 6070 sys_catalog.cc:455] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cefeac210b6047d38756f5f96a43f3e8. Latest consensus state: current_term: 2 leader_uuid: "cefeac210b6047d38756f5f96a43f3e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } } }
I20260928 14:41:11.825677 6070 sys_catalog.cc:458] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:11.825816 6075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:41:11.826855 6075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:41:11.827144 6075 catalog_manager.cc:1259] Loaded cluster ID: 0ec02827e4f04bc5af6a11056249af72
I20260928 14:41:11.827260 6075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:41:11.828090 6075 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:41:11.828302 6075 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Loaded TSK: 0
I20260928 14:41:11.847101 6075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Generated new TSK 1
I20260928 14:41:11.847246 6075 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 14:41:11.909425 6066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:11.909660 6066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:11.909689 6066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:11.911011 6066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:11.911065 6066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:11.911096 6066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:41:11.912698 6066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:38419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=38157
--tserver_master_addrs=127.22.13.126:41223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:11.912976 6066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:11.913261 6066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:11.919711 6091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:11.919859 6066 server_base.cc:1034] running on GCE node
W20260928 14:41:11.919735 6094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:11.919736 6092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:11.920274 6066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:11.920575 6066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:11.921753 6066 hybrid_clock.cc:648] HybridClock initialized: now 1790606471921735 us; error 39 us; skew 500 ppm
Sep 28 14:41:11 dist-test-slave-wwwb krb5kdc[5449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606471, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:11.924546 6066 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:41:11.925252 6066 webserver.cc:460] Webserver started at http://127.22.13.65:38157/ using document root <none> and password file <none>
I20260928 14:41:11.925534 6066 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:11.925637 6066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:11.927424 6066 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:11.928300 6101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:11.928473 6066 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:11.928583 6066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "22e98f4d9dd9467c92809fdd2c8eaf79"
format_stamp: "Formatted at 2026-09-28 14:40:50 on dist-test-slave-wwwb"
server_key: "788f8365bf2825945ac3fdd2498901c1"
server_key_iv: "9d7a5e65dcb62dce5aab49e6389d1dd5"
server_key_version: "encryptionkey@0"
I20260928 14:41:11.928973 6066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:11.952369 6066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:11.954748 6066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:11.954926 6066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:11.955185 6066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:11.955749 6108 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 14:41:11.957279 6066 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 14:41:11.957336 6066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 14:41:11.957384 6066 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 14:41:11.958158 6066 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 14:41:11.958205 6066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:11.958249 6108 tablet_bootstrap.cc:492] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: Bootstrap starting.
I20260928 14:41:11.966528 6066 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:38419
I20260928 14:41:11.966600 6215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:38419 every 8 connection(s)
I20260928 14:41:11.967281 6066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:41:11.970381 6108 log.cc:824] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:11.972625 6108 tablet_bootstrap.cc:492] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:41:11.973157 6108 tablet_bootstrap.cc:492] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: Bootstrap complete.
I20260928 14:41:11.974439 6108 ts_tablet_manager.cc:1397] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.012s
I20260928 14:41:11.975673 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 6066
I20260928 14:41:11.975818 6108 raft_consensus.cc:348] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } } peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } }
I20260928 14:41:11.976241 6108 raft_consensus.cc:729] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22e98f4d9dd9467c92809fdd2c8eaf79, State: Initialized, Role: FOLLOWER
I20260928 14:41:11.976325 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:41107
--local_ip_for_outbound_sockets=127.22.13.66
--tserver_master_addrs=127.22.13.126:41223
--webserver_port=37605
--webserver_interface=127.22.13.66
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:41:11.976409 6108 consensus_queue.cc:260] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } } peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } }
I20260928 14:41:11.976727 6108 ts_tablet_manager.cc:1428] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
Sep 28 14:41:11 dist-test-slave-wwwb krb5kdc[5449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606471, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:41:11.982956 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:11.968190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:42749 (local address 127.22.13.126:41223)
0928 14:41:11.969359 (+ 1169us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:11.969369 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:11.969411 (+ 42us) server_negotiation.cc:407] Connection header received
0928 14:41:11.969556 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:11.969562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:11.969619 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:11.969758 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:11.970747 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:11.972580 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:11.974598 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:11.975209 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:11.975336 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:11.978462 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:11.978468 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:11.978468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:11.978482 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:11.980616 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:11.981253 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:11.981256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:11.981257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:11.981403 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:11.981754 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:11.981755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:11.981756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:11.982111 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:11.982204 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:11.982610 (+ 406us) server_negotiation.cc:299] Negotiation successful
0928 14:41:11.982755 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1110,"thread_start_us":83,"threads_started":1}
I20260928 14:41:11.983026 6218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:11.968037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:42749)
0928 14:41:11.968318 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:11.968366 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:11.968994 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:11.969839 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:11.969845 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:11.970301 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:11.970603 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:11.970611 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:11.972923 (+ 2312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:11.972924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:11.974433 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:11.974437 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:11.975347 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:11.975348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:11.975549 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:11.976170 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:41:11.976181 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:11.978324 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:11.980744 (+ 2420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:11.980755 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:11.980758 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:11.981138 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:11.981492 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:11.981493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:11.981494 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:11.981658 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:11.982195 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:11.982203 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:11.982392 (+ 189us) client_negotiation.cc:769] Sending connection context
0928 14:41:11.982515 (+ 123us) client_negotiation.cc:240] Negotiation successful
0928 14:41:11.982737 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":95,"threads_started":1}
I20260928 14:41:11.984390 6216 heartbeater.cc:347] Connected to a master server at 127.22.13.126:41223
I20260928 14:41:11.984520 6216 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:11.984752 6216 heartbeater.cc:511] Master 127.22.13.126:41223 requested a full tablet report, sending...
I20260928 14:41:11.985615 6028 ts_manager.cc:194] Registered new tserver with Master: 22e98f4d9dd9467c92809fdd2c8eaf79 (127.22.13.65:38419)
I20260928 14:41:11.986925 6028 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:42749
I20260928 14:41:11.988055 6216 heartbeater.cc:503] Master 127.22.13.126:41223 was elected leader, sending a full tablet report...
Sep 28 14:41:11 dist-test-slave-wwwb krb5kdc[5449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606471, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for kudu/127.22.13.65@KRBTEST.COM
I20260928 14:41:11.996373 6223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:11.986359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38419 (local address 127.0.0.1:51470)
0928 14:41:11.986580 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:11.986629 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:11.986709 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:11.987695 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:11.987702 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:11.988087 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:11.988159 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:11.988163 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:11.989490 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:11.989491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:11.990359 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:11.990362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:11.991024 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:11.991025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:11.991090 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:11.991626 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:41:11.991630 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:11.993193 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:11.995262 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:11.995269 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:11.995271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:11.995479 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:11.995699 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:11.995700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:11.995700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:11.995757 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:11.996062 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:11.996068 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:11.996113 (+ 45us) client_negotiation.cc:769] Sending connection context
0928 14:41:11.996179 (+ 66us) client_negotiation.cc:240] Negotiation successful
0928 14:41:11.996290 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":119,"threads_started":1}
I20260928 14:41:11.996420 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:11.987287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51470 (local address 127.22.13.65:38419)
0928 14:41:11.987509 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:11.987515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:11.987527 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:41:11.987568 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:11.987571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:11.987589 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:11.987672 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:11.988258 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:11.989392 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:11.990445 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:11.990939 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:11.991023 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:11.993292 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:11.993302 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:11.993302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:11.993311 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:11.995170 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:11.995562 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:11.995565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:11.995566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:11.995633 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:11.995812 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:11.995812 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:11.995813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:11.995998 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:11.996082 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:11.996232 (+ 150us) server_negotiation.cc:299] Negotiation successful
0928 14:41:11.996317 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":128,"threads_started":1}
I20260928 14:41:11.996923 6150 tablet_service.cc:1514] Processing DeleteTablet for tablet b196aa0b46364c2f9801926c8574e3e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:40:51 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:51470
I20260928 14:41:11.997553 6225 tablet_replica.cc:330] stopping tablet replica
I20260928 14:41:11.997687 6225 raft_consensus.cc:2229] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:41:11.997776 6225 raft_consensus.cc:2258] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:11.998144 6225 ts_tablet_manager.cc:1905] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:12.000317 6225 ts_tablet_manager.cc:1918] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:41:12.000417 6225 log.cc:1196] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal/wals/b196aa0b46364c2f9801926c8574e3e1
I20260928 14:41:12.000715 6225 ts_tablet_manager.cc:1939] T b196aa0b46364c2f9801926c8574e3e1 P 22e98f4d9dd9467c92809fdd2c8eaf79: Deleting consensus metadata
I20260928 14:41:12.001279 6012 catalog_manager.cc:4841] TS 22e98f4d9dd9467c92809fdd2c8eaf79 (127.22.13.65:38419): tablet b196aa0b46364c2f9801926c8574e3e1 (table test-table [id=a7cb9058aab048b8bfcb1face5aec0ff]) successfully deleted
W20260928 14:41:12.069036 6222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:12.069269 6222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:12.069296 6222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:12.070571 6222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:12.070628 6222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:12.070658 6222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:41:12.072463 6222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:41107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=37605
--tserver_master_addrs=127.22.13.126:41223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:12.072742 6222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:12.073040 6222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:12.078809 6231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:12.078832 6230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:12.078840 6233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:12.079231 6222 server_base.cc:1034] running on GCE node
I20260928 14:41:12.079516 6222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:12.079802 6222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:12.080981 6222 hybrid_clock.cc:648] HybridClock initialized: now 1790606472080962 us; error 28 us; skew 500 ppm
Sep 28 14:41:12 dist-test-slave-wwwb krb5kdc[5449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606472, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:12.083880 6222 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:41:12.084422 6222 webserver.cc:460] Webserver started at http://127.22.13.66:37605/ using document root <none> and password file <none>
I20260928 14:41:12.084659 6222 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:12.084721 6222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:12.086354 6222 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:12.087267 6240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:12.087455 6222 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:41:12.087545 6222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "ee9c8b5cc3ac444ab1551d1634c47e82"
format_stamp: "Formatted at 2026-09-28 14:40:51 on dist-test-slave-wwwb"
server_key: "dbc961026bc0aeac41e9429e2a575cc8"
server_key_iv: "6ecadece95371e6d67bfb086695bc318"
server_key_version: "encryptionkey@0"
I20260928 14:41:12.087906 6222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:12.100495 6222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:12.102607 6222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:12.102731 6222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:12.102932 6222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:12.103271 6222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:12.103304 6222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:12.103327 6222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:12.103374 6222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:12.109308 6222 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:41107
I20260928 14:41:12.109417 6353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:41107 every 8 connection(s)
I20260928 14:41:12.109777 6222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:41:12.112536 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 6222
I20260928 14:41:12.112869 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:37733
--local_ip_for_outbound_sockets=127.22.13.67
--tserver_master_addrs=127.22.13.126:41223
--webserver_port=33409
--webserver_interface=127.22.13.67
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:41:12 dist-test-slave-wwwb krb5kdc[5449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606472, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:41:12.121130 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.110630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:49557 (local address 127.22.13.126:41223)
0928 14:41:12.110698 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:12.110700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:12.110963 (+ 263us) server_negotiation.cc:407] Connection header received
0928 14:41:12.111528 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:12.111530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:12.111542 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:12.111590 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:12.112781 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.113398 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.114773 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.114984 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.115062 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:12.117925 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:12.117926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:12.117926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:12.117974 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:12.119448 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:12.120045 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:12.120045 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:12.120046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:12.120098 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:12.120419 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:12.120420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:12.120420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:12.120530 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:12.120580 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:12.120976 (+ 396us) server_negotiation.cc:299] Negotiation successful
0928 14:41:12.121062 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:41:12.121595 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.110621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:49557)
0928 14:41:12.110811 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:12.110850 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:12.111403 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:12.111672 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:12.111676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:12.112288 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:12.112653 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:12.112660 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.113506 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.113507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:12.114671 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:12.114674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.115085 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.115086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:12.115208 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:12.115781 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:41:12.115790 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:12.117809 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:12.119550 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:12.119560 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:12.119563 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:12.119951 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:12.120168 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:12.120169 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:12.120170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:12.120346 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:12.120600 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:12.120608 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:12.120821 (+ 213us) client_negotiation.cc:769] Sending connection context
0928 14:41:12.120968 (+ 147us) client_negotiation.cc:240] Negotiation successful
0928 14:41:12.121187 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":96,"threads_started":1}
I20260928 14:41:12.122215 6354 heartbeater.cc:347] Connected to a master server at 127.22.13.126:41223
I20260928 14:41:12.122337 6354 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:12.122592 6354 heartbeater.cc:511] Master 127.22.13.126:41223 requested a full tablet report, sending...
I20260928 14:41:12.123077 6028 ts_manager.cc:194] Registered new tserver with Master: ee9c8b5cc3ac444ab1551d1634c47e82 (127.22.13.66:41107)
I20260928 14:41:12.123468 6028 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:49557
W20260928 14:41:12.204344 6357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:12.204697 6357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:12.204754 6357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:12.206221 6357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:12.206290 6357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:12.206339 6357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:41:12.207907 6357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:37733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=33409
--tserver_master_addrs=127.22.13.126:41223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:12.208204 6357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:12.208446 6357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:12.213505 6365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:12.213518 6362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:12.213531 6363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:12.213795 6357 server_base.cc:1034] running on GCE node
I20260928 14:41:12.214046 6357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:12.214340 6357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:12.215512 6357 hybrid_clock.cc:648] HybridClock initialized: now 1790606472215502 us; error 36 us; skew 500 ppm
Sep 28 14:41:12 dist-test-slave-wwwb krb5kdc[5449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606472, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:12.218330 6357 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:41:12.218757 6357 webserver.cc:460] Webserver started at http://127.22.13.67:33409/ using document root <none> and password file <none>
I20260928 14:41:12.218956 6357 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:12.219013 6357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:12.220490 6357 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:12.221095 6372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:12.221280 6357 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:12.221382 6357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "a0ed8637b495448fbf2edae1c8aa4161"
format_stamp: "Formatted at 2026-09-28 14:40:51 on dist-test-slave-wwwb"
server_key: "23972ce1adb883570ab5bd70f174a4bf"
server_key_iv: "bd1ce638133273522b782978b551c787"
server_key_version: "encryptionkey@0"
I20260928 14:41:12.221766 6357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:12.239262 6357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:12.241369 6357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:12.241544 6357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:12.241791 6357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:12.242158 6357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:12.242213 6357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:12.242257 6357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:12.242291 6357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:12.248231 6357 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:37733
I20260928 14:41:12.248276 6485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:37733 every 8 connection(s)
I20260928 14:41:12.248659 6357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
Sep 28 14:41:12 dist-test-slave-wwwb krb5kdc[5449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606472, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:41:12.258121 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.249403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:42717 (local address 127.22.13.126:41223)
0928 14:41:12.249478 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:12.249480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:12.249705 (+ 225us) server_negotiation.cc:407] Connection header received
0928 14:41:12.250271 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:12.250272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:12.250282 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:12.250339 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:12.251332 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.251822 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.252856 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.253068 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.253129 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:12.255467 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:12.255469 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:12.255469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:12.255484 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:12.256754 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:12.257196 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:12.257197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:12.257197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:12.257249 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:12.257487 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:12.257488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:12.257488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:12.257598 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:12.257649 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:12.257909 (+ 260us) server_negotiation.cc:299] Negotiation successful
0928 14:41:12.258051 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 14:41:12.258328 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.249397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:42717)
0928 14:41:12.249572 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:12.249610 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:12.250172 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:12.250420 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:12.250423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:12.251020 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:12.251235 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:12.251239 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.251911 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.251912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:12.252749 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:12.252751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.253153 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.253154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:12.253265 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:12.253780 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:41:12.253789 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:12.255366 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:12.256847 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:12.256855 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:12.256858 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:12.257122 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:12.257311 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:12.257312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:12.257312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:12.257423 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:12.257656 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:12.257660 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:12.257793 (+ 133us) client_negotiation.cc:769] Sending connection context
0928 14:41:12.257884 (+ 91us) client_negotiation.cc:240] Negotiation successful
0928 14:41:12.258097 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260928 14:41:12.258668 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 6357
I20260928 14:41:12.258855 6486 heartbeater.cc:347] Connected to a master server at 127.22.13.126:41223
I20260928 14:41:12.258976 6486 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:12.259188 6486 heartbeater.cc:511] Master 127.22.13.126:41223 requested a full tablet report, sending...
I20260928 14:41:12.259640 6028 ts_manager.cc:194] Registered new tserver with Master: a0ed8637b495448fbf2edae1c8aa4161 (127.22.13.67:37733)
I20260928 14:41:12.260038 6028 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:42717
I20260928 14:41:12.265277 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.259093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40042 (local address 127.22.13.126:41223)
0928 14:41:12.259138 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:12.259140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:12.259291 (+ 151us) server_negotiation.cc:407] Connection header received
0928 14:41:12.259357 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:12.259358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:12.259370 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:12.259429 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:12.260113 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.260629 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.261408 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.261617 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.261695 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:12.262940 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:12.262942 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:12.262942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:12.262950 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:12.264085 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:12.264449 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:12.264450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:12.264450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:12.264510 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:12.264732 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:12.264733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:12.264733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:12.264853 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:12.265057 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:12.265095 (+ 38us) server_negotiation.cc:299] Negotiation successful
0928 14:41:12.265171 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:41:12.266202 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:41:12.269202 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.266562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40058 (local address 127.22.13.126:41223)
0928 14:41:12.266646 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:12.266648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:12.266759 (+ 111us) server_negotiation.cc:407] Connection header received
0928 14:41:12.266849 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:12.266850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:12.266892 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:12.266988 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:41:12.267123 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.267606 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.268378 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.268592 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.268669 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:12.268851 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:41:12.268914 (+ 63us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0928 14:41:12.269127 (+ 213us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40058: token expired
Metrics: {"server-negotiator.queue_time_us":39}
W20260928 14:41:12.269270 6219 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40058: token expired
W20260928 14:41:12.269354 5958 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.13.126:41223)
I20260928 14:41:12.275775 6219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.269608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40072 (local address 127.22.13.126:41223)
0928 14:41:12.269647 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:12.269649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:12.269681 (+ 32us) server_negotiation.cc:407] Connection header received
0928 14:41:12.269741 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:12.269741 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:12.269749 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:12.269785 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:12.270315 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.270764 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.271437 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.271612 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.271662 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:12.272817 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:12.272819 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:12.272819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:12.272863 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:12.274645 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:12.275026 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:12.275027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:12.275027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:12.275087 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:12.275260 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:12.275261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:12.275261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:12.275378 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:12.275470 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:12.275570 (+ 100us) server_negotiation.cc:299] Negotiation successful
0928 14:41:12.275641 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:41:12.277060 6028 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:41:12.278821 6028 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:41:12.281255 6150 tablet_service.cc:1467] Processing CreateTablet for tablet c30359ab370f4623bb6e59ec9af1137d (DEFAULT_TABLE table=test-table [id=2c8fb48573d14d35973e506ddce26788]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:12.281493 6150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c30359ab370f4623bb6e59ec9af1137d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:12.282951 6108 tablet_bootstrap.cc:492] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79: Bootstrap starting.
I20260928 14:41:12.283311 6108 tablet_bootstrap.cc:654] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:12.283972 6108 tablet_bootstrap.cc:492] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79: No bootstrap required, opened a new log
I20260928 14:41:12.284032 6108 ts_tablet_manager.cc:1397] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260928 14:41:12.284389 6108 raft_consensus.cc:348] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } } peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } }
I20260928 14:41:12.284474 6108 raft_consensus.cc:374] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:12.284535 6108 raft_consensus.cc:729] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22e98f4d9dd9467c92809fdd2c8eaf79, State: Initialized, Role: FOLLOWER
I20260928 14:41:12.284585 6108 consensus_queue.cc:260] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } } peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } }
I20260928 14:41:12.284698 6108 ts_tablet_manager.cc:1428] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260928 14:41:12.285563 6491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.280983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:41107 (local address 127.0.0.1:39574)
0928 14:41:12.281236 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:12.281247 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:12.281299 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:12.281709 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:12.281710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:12.281712 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:12.281750 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:12.281751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.283136 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.283138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:12.284278 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:12.284282 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.285278 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.285279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:12.285350 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:12.285380 (+ 30us) client_negotiation.cc:769] Sending connection context
0928 14:41:12.285433 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 14:41:12.285483 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":89,"threads_started":1}
I20260928 14:41:12.285784 6223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.280917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:37733 (local address 127.0.0.1:58690)
0928 14:41:12.280983 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:12.280999 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:12.281049 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:12.281497 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:12.281498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:12.281501 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:12.281567 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:12.281569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.283172 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.283172 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:12.284338 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:12.284341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.285518 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.285519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:12.285606 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:12.285610 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:41:12.285662 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 14:41:12.285712 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260928 14:41:12.285844 6493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.281242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39574 (local address 127.22.13.66:41107)
0928 14:41:12.281450 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:12.281454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:12.281463 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:41:12.281504 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:12.281506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:12.281521 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:12.281603 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:12.281854 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.283020 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.284397 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.285177 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.285268 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:12.285348 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:12.285704 (+ 356us) server_negotiation.cc:299] Negotiation successful
0928 14:41:12.285751 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":94,"threads_started":1}
I20260928 14:41:12.286156 6492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.281035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58690 (local address 127.22.13.67:37733)
0928 14:41:12.281249 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:12.281255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:12.281268 (+ 13us) server_negotiation.cc:407] Connection header received
0928 14:41:12.281319 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:12.281321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:12.281356 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:12.281652 (+ 296us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:12.281746 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.283056 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.284528 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.285326 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.285396 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:12.285484 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:12.286028 (+ 544us) server_negotiation.cc:299] Negotiation successful
0928 14:41:12.286081 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":115,"threads_started":1}
I20260928 14:41:12.286705 6288 tablet_service.cc:1467] Processing CreateTablet for tablet c30359ab370f4623bb6e59ec9af1137d (DEFAULT_TABLE table=test-table [id=2c8fb48573d14d35973e506ddce26788]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:12.286761 6420 tablet_service.cc:1467] Processing CreateTablet for tablet c30359ab370f4623bb6e59ec9af1137d (DEFAULT_TABLE table=test-table [id=2c8fb48573d14d35973e506ddce26788]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:12.287111 6288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c30359ab370f4623bb6e59ec9af1137d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:12.287111 6420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c30359ab370f4623bb6e59ec9af1137d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:12.291172 6496 tablet_bootstrap.cc:492] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82: Bootstrap starting.
I20260928 14:41:12.291908 6496 tablet_bootstrap.cc:654] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:12.292490 6496 log.cc:824] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:12.292831 6495 tablet_bootstrap.cc:492] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161: Bootstrap starting.
I20260928 14:41:12.293391 6496 tablet_bootstrap.cc:492] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82: No bootstrap required, opened a new log
I20260928 14:41:12.293478 6496 ts_tablet_manager.cc:1397] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:41:12.293530 6495 tablet_bootstrap.cc:654] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:12.294128 6495 log.cc:824] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:12.294952 6495 tablet_bootstrap.cc:492] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161: No bootstrap required, opened a new log
I20260928 14:41:12.295029 6495 ts_tablet_manager.cc:1397] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260928 14:41:12.295732 6496 raft_consensus.cc:348] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } } peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } }
I20260928 14:41:12.295940 6496 raft_consensus.cc:374] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:12.295989 6496 raft_consensus.cc:729] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee9c8b5cc3ac444ab1551d1634c47e82, State: Initialized, Role: FOLLOWER
I20260928 14:41:12.296119 6496 consensus_queue.cc:260] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } } peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } }
I20260928 14:41:12.296425 6496 ts_tablet_manager.cc:1428] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:41:12.296434 6354 heartbeater.cc:503] Master 127.22.13.126:41223 was elected leader, sending a full tablet report...
I20260928 14:41:12.297106 6495 raft_consensus.cc:348] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } } peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } }
I20260928 14:41:12.297330 6495 raft_consensus.cc:374] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:12.297394 6495 raft_consensus.cc:729] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ed8637b495448fbf2edae1c8aa4161, State: Initialized, Role: FOLLOWER
I20260928 14:41:12.297516 6495 consensus_queue.cc:260] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } } peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } }
I20260928 14:41:12.297792 6486 heartbeater.cc:503] Master 127.22.13.126:41223 was elected leader, sending a full tablet report...
I20260928 14:41:12.297799 6495 ts_tablet_manager.cc:1428] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260928 14:41:12.360427 6355 tablet.cc:2369] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:41:12.394433 6221 raft_consensus.cc:482] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:41:12.394613 6221 raft_consensus.cc:504] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } } peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } }
I20260928 14:41:12.395054 6221 leader_election.cc:288] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0ed8637b495448fbf2edae1c8aa4161 (127.22.13.67:37733), ee9c8b5cc3ac444ab1551d1634c47e82 (127.22.13.66:41107)
I20260928 14:41:12.398283 6493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.395380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:39323 (local address 127.22.13.66:41107)
0928 14:41:12.395423 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:12.395425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:12.395611 (+ 186us) server_negotiation.cc:407] Connection header received
0928 14:41:12.395650 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:12.395651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:12.395665 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:12.395731 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:12.395944 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.396420 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.397455 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.397842 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.397957 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:12.397966 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:12.398170 (+ 204us) server_negotiation.cc:299] Negotiation successful
0928 14:41:12.398214 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:41:12.398293 6218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.395255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:37733 (local address 127.22.13.65:44799)
0928 14:41:12.395321 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:12.395335 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:12.395404 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:12.395623 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:12.395624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:12.395627 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:12.395711 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:12.395714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.396446 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.396448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:12.397319 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:12.397321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.397915 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.397916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:12.398032 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:12.398066 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 14:41:12.398116 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 14:41:12.398187 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":31}
I20260928 14:41:12.398278 6492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.395342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:44799 (local address 127.22.13.67:37733)
0928 14:41:12.395394 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:12.395397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:12.395410 (+ 13us) server_negotiation.cc:407] Connection header received
0928 14:41:12.395488 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:12.395489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:12.395500 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:12.395574 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:12.395797 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.396347 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.397432 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.397824 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.397885 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:12.397892 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:12.398144 (+ 252us) server_negotiation.cc:299] Negotiation successful
0928 14:41:12.398194 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:41:12.398401 6501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.395279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:41107 (local address 127.22.13.65:39323)
0928 14:41:12.395521 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:12.395532 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:12.395597 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:12.395779 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:12.395779 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:12.395781 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:12.395863 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:12.395865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.396519 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.396520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:12.397319 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:12.397322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.398018 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.398019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:12.398073 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:12.398077 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:41:12.398152 (+ 75us) client_negotiation.cc:240] Negotiation successful
0928 14:41:12.398194 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":49,"thread_start_us":118,"threads_started":1}
I20260928 14:41:12.398700 6440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c30359ab370f4623bb6e59ec9af1137d" candidate_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ed8637b495448fbf2edae1c8aa4161" is_pre_election: true
I20260928 14:41:12.398788 6308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c30359ab370f4623bb6e59ec9af1137d" candidate_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" is_pre_election: true
I20260928 14:41:12.398900 6440 raft_consensus.cc:2454] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22e98f4d9dd9467c92809fdd2c8eaf79 in term 0.
I20260928 14:41:12.398968 6308 raft_consensus.cc:2454] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22e98f4d9dd9467c92809fdd2c8eaf79 in term 0.
I20260928 14:41:12.399163 6102 leader_election.cc:302] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22e98f4d9dd9467c92809fdd2c8eaf79, a0ed8637b495448fbf2edae1c8aa4161; no voters:
I20260928 14:41:12.399336 6221 raft_consensus.cc:2781] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:41:12.399422 6221 raft_consensus.cc:482] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:41:12.399448 6221 raft_consensus.cc:3037] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:12.400254 6221 raft_consensus.cc:504] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } } peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } }
I20260928 14:41:12.400415 6221 leader_election.cc:288] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [CANDIDATE]: Term 1 election: Requested vote from peers a0ed8637b495448fbf2edae1c8aa4161 (127.22.13.67:37733), ee9c8b5cc3ac444ab1551d1634c47e82 (127.22.13.66:41107)
I20260928 14:41:12.400576 6440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c30359ab370f4623bb6e59ec9af1137d" candidate_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ed8637b495448fbf2edae1c8aa4161"
I20260928 14:41:12.400616 6308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c30359ab370f4623bb6e59ec9af1137d" candidate_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82"
I20260928 14:41:12.400673 6440 raft_consensus.cc:3037] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:12.400715 6308 raft_consensus.cc:3037] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:12.401422 6308 raft_consensus.cc:2454] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22e98f4d9dd9467c92809fdd2c8eaf79 in term 1.
I20260928 14:41:12.401486 6440 raft_consensus.cc:2454] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22e98f4d9dd9467c92809fdd2c8eaf79 in term 1.
I20260928 14:41:12.401671 6103 leader_election.cc:302] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22e98f4d9dd9467c92809fdd2c8eaf79, ee9c8b5cc3ac444ab1551d1634c47e82; no voters:
I20260928 14:41:12.401789 6221 raft_consensus.cc:2781] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:41:12.402015 6221 raft_consensus.cc:686] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 1 LEADER]: Becoming Leader. State: Replica: 22e98f4d9dd9467c92809fdd2c8eaf79, State: Running, Role: LEADER
I20260928 14:41:12.402109 6221 consensus_queue.cc:237] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } } peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } }
I20260928 14:41:12.402904 6028 catalog_manager.cc:5495] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22e98f4d9dd9467c92809fdd2c8eaf79 (127.22.13.65). New cstate: current_term: 1 leader_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22e98f4d9dd9467c92809fdd2c8eaf79" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38419 } health_report { overall_health: HEALTHY } } }
I20260928 14:41:12.433027 6224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.430518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51486 (local address 127.22.13.65:38419)
0928 14:41:12.430581 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:12.430583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:12.430590 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:41:12.430629 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:12.430630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:12.430647 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:12.430712 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:41:12.430829 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.431332 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.432130 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:12.432374 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:12.432455 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:12.432646 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:41:12.432768 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:41:12.432851 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:12.432904 (+ 53us) server_negotiation.cc:299] Negotiation successful
0928 14:41:12.432933 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:41:12.434654 6440 raft_consensus.cc:1261] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161 [term 1 FOLLOWER]: Refusing update from remote peer 22e98f4d9dd9467c92809fdd2c8eaf79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:41:12.434654 6308 raft_consensus.cc:1261] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82 [term 1 FOLLOWER]: Refusing update from remote peer 22e98f4d9dd9467c92809fdd2c8eaf79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:41:12.435065 6502 consensus_queue.cc:1035] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee9c8b5cc3ac444ab1551d1634c47e82" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 41107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:41:12.435204 6221 consensus_queue.cc:1035] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ed8637b495448fbf2edae1c8aa4161" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:41:12.437714 6504 mvcc.cc:205] Tried to move back new op lower bound from 7334324111089860608 to 7334324110959624192. Current Snapshot: MvccSnapshot[applied={T|T < 7334324111089860608}]
I20260928 14:41:12.440444 6026 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:41:12.440560 6026 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:41:12.441315 6026 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Sending DeleteTablet for 3 replicas of tablet c30359ab370f4623bb6e59ec9af1137d
I20260928 14:41:12.441645 6150 tablet_service.cc:1514] Processing DeleteTablet for tablet c30359ab370f4623bb6e59ec9af1137d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:12 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:51470
I20260928 14:41:12.441651 6420 tablet_service.cc:1514] Processing DeleteTablet for tablet c30359ab370f4623bb6e59ec9af1137d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:12 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:58690
I20260928 14:41:12.441762 6225 tablet_replica.cc:330] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79: stopping tablet replica
I20260928 14:41:12.441816 6225 raft_consensus.cc:2229] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:41:12.441790 6288 tablet_service.cc:1514] Processing DeleteTablet for tablet c30359ab370f4623bb6e59ec9af1137d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:12 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:39574
I20260928 14:41:12.441907 6225 raft_consensus.cc:2258] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:12.442058 6514 tablet_replica.cc:330] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161: stopping tablet replica
I20260928 14:41:12.442059 6515 tablet_replica.cc:330] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82: stopping tablet replica
I20260928 14:41:12.442154 6514 raft_consensus.cc:2229] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:41:12.442164 6515 raft_consensus.cc:2229] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:41:12.442265 6515 raft_consensus.cc:2258] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:12.442296 6514 raft_consensus.cc:2258] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:12.442481 6225 ts_tablet_manager.cc:1905] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:12.442894 6514 ts_tablet_manager.cc:1905] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:12.443802 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 6066
I20260928 14:41:12.445076 6514 ts_tablet_manager.cc:1918] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:41:12.445144 6514 log.cc:1196] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/c30359ab370f4623bb6e59ec9af1137d
I20260928 14:41:12.445428 6514 ts_tablet_manager.cc:1939] T c30359ab370f4623bb6e59ec9af1137d P a0ed8637b495448fbf2edae1c8aa4161: Deleting consensus metadata
I20260928 14:41:12.446005 6012 catalog_manager.cc:4841] TS a0ed8637b495448fbf2edae1c8aa4161 (127.22.13.67:37733): tablet c30359ab370f4623bb6e59ec9af1137d (table test-table [id=2c8fb48573d14d35973e506ddce26788]) successfully deleted
W20260928 14:41:12.450132 6012 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:38419 (error 108)
I20260928 14:41:12.450515 6515 ts_tablet_manager.cc:1905] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260928 14:41:12.450692 6223 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:41:12.450570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38419 (local address 127.0.0.1:51490)
0928 14:41:12.450613 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:12.450651 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:38419: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":20}
W20260928 14:41:12.450811 6012 catalog_manager.cc:4584] TS 22e98f4d9dd9467c92809fdd2c8eaf79 (127.22.13.65:38419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c30359ab370f4623bb6e59ec9af1137d: Network error: Client connection negotiation failed: client connection to 127.22.13.65:38419: connect: Connection reset by peer (error 104)
I20260928 14:41:12.451485 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 6222
I20260928 14:41:12.456051 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 6357
I20260928 14:41:12.456279 6223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:12.456111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:41107 (local address 127.0.0.1:39578)
0928 14:41:12.456191 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:12.456231 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.66:41107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260928 14:41:12.460769 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 5995
I20260928 14:41:32.468329 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41223
--webserver_interface=127.22.13.126
--webserver_port=42451
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:41223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:32.550184 6517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:32.550486 6517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:32.550539 6517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:32.551884 6517 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 14:41:32.551982 6517 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 14:41:32.552021 6517 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 14:41:32.552042 6517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:41:32.552078 6517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:32.552124 6517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:32.552152 6517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:41:32.552178 6517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:41:32.553969 6517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:41223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=42451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:32.554283 6517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:32.554541 6517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:32.559540 6525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:32.559556 6523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:32.559854 6517 server_base.cc:1034] running on GCE node
W20260928 14:41:32.559563 6522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:32.560160 6517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:32.560479 6517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:32.561637 6517 hybrid_clock.cc:648] HybridClock initialized: now 1790606492561616 us; error 38 us; skew 500 ppm
Sep 28 14:41:32 dist-test-slave-wwwb krb5kdc[5449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606492, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:32.568010 6517 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:41:32.568554 6517 webserver.cc:460] Webserver started at http://127.22.13.126:42451/ using document root <none> and password file <none>
I20260928 14:41:32.568818 6517 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:32.568902 6517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:32.570412 6517 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:41:32.571152 6532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:32.571329 6517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:32.571444 6517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "cefeac210b6047d38756f5f96a43f3e8"
format_stamp: "Formatted at 2026-09-28 14:40:50 on dist-test-slave-wwwb"
server_key: "0e2f7b6824feffc425bd809aa1929e2f"
server_key_iv: "08adbafdd7a9c704f63850c1f1c1bd70"
server_key_version: "encryptionkey@0"
I20260928 14:41:32.571831 6517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:32.588744 6517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:32.590950 6517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:32.591162 6517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:32.595643 6517 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:41223
I20260928 14:41:32.595681 6584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:41223 every 8 connection(s)
I20260928 14:41:32.596100 6517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:41:32.597628 6585 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:41:32.598402 6585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Bootstrap starting.
I20260928 14:41:32.604388 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 6517
I20260928 14:41:32.604758 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:38419
--local_ip_for_outbound_sockets=127.22.13.65
--tserver_master_addrs=127.22.13.126:41223
--webserver_port=38157
--webserver_interface=127.22.13.65
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:41:32.607666 6585 log.cc:824] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:32.610237 6585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:41:32.610591 6585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Bootstrap complete.
I20260928 14:41:32.612900 6585 raft_consensus.cc:348] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } }
I20260928 14:41:32.613137 6585 raft_consensus.cc:729] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: cefeac210b6047d38756f5f96a43f3e8, State: Initialized, Role: FOLLOWER
I20260928 14:41:32.613304 6585 consensus_queue.cc:260] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } }
I20260928 14:41:32.613381 6585 raft_consensus.cc:388] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:41:32.613451 6585 raft_consensus.cc:482] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:41:32.613497 6585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 2 FOLLOWER]: Advancing to term 3
I20260928 14:41:32.614640 6585 raft_consensus.cc:504] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } }
I20260928 14:41:32.614734 6585 leader_election.cc:302] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cefeac210b6047d38756f5f96a43f3e8; no voters:
I20260928 14:41:32.614954 6585 leader_election.cc:288] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260928 14:41:32.614997 6589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260928 14:41:32.615195 6589 raft_consensus.cc:686] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [term 3 LEADER]: Becoming Leader. State: Replica: cefeac210b6047d38756f5f96a43f3e8, State: Running, Role: LEADER
I20260928 14:41:32.615245 6585 sys_catalog.cc:564] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:41:32.615355 6589 consensus_queue.cc:237] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } }
I20260928 14:41:32.615759 6590 sys_catalog.cc:455] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "cefeac210b6047d38756f5f96a43f3e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } } }
I20260928 14:41:32.615852 6590 sys_catalog.cc:458] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:32.616087 6591 sys_catalog.cc:455] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cefeac210b6047d38756f5f96a43f3e8. Latest consensus state: current_term: 3 leader_uuid: "cefeac210b6047d38756f5f96a43f3e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cefeac210b6047d38756f5f96a43f3e8" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41223 } } }
I20260928 14:41:32.616181 6591 sys_catalog.cc:458] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:32.616434 6597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:41:32.617370 6597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:41:32.617522 6597 catalog_manager.cc:1259] Loaded cluster ID: 0ec02827e4f04bc5af6a11056249af72
I20260928 14:41:32.617589 6597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:41:32.618891 6597 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:41:32.619135 6597 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Loaded TSK: 0
I20260928 14:41:32.619242 6597 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cefeac210b6047d38756f5f96a43f3e8: Loaded TSK: 1
F20260928 14:41:32.619550 6597 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7feab83a69ad google::LogMessage::Fail() at ??:0
@ 0x7feab83a988a google::LogMessage::SendToLog() at ??:0
@ 0x7feab83a64de google::LogMessage::Flush() at ??:0
@ 0x7feab83a7a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7feabab13104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7feabab10d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7feac03001cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7feac030048e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7feac0300594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7feac02c6c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7feac02d0ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7feaba2740d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7feaba269954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7feabb0cd6db start_thread at ??:0
@ 0x7feab6ea461f clone at ??:0
W20260928 14:41:32.709127 6587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:32.709365 6587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:32.709394 6587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:32.710734 6587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:32.710788 6587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:32.710819 6587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:41:32.712421 6587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:38419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=38157
--tserver_master_addrs=127.22.13.126:41223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:32.712709 6587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:32.712973 6587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:32.718314 6627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:32.718283 6626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:32.718274 6629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:32.718734 6587 server_base.cc:1034] running on GCE node
I20260928 14:41:32.718961 6587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:32.719225 6587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:32.720391 6587 hybrid_clock.cc:648] HybridClock initialized: now 1790606492720376 us; error 32 us; skew 500 ppm
Sep 28 14:41:32 dist-test-slave-wwwb krb5kdc[5449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606492, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:32.723155 6587 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:41:32.723662 6587 webserver.cc:460] Webserver started at http://127.22.13.65:38157/ using document root <none> and password file <none>
I20260928 14:41:32.723902 6587 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:32.723970 6587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:32.725450 6587 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:32.726262 6636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:32.726462 6587 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:32.726591 6587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "22e98f4d9dd9467c92809fdd2c8eaf79"
format_stamp: "Formatted at 2026-09-28 14:40:50 on dist-test-slave-wwwb"
server_key: "788f8365bf2825945ac3fdd2498901c1"
server_key_iv: "9d7a5e65dcb62dce5aab49e6389d1dd5"
server_key_version: "encryptionkey@0"
I20260928 14:41:32.726967 6587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:32.739941 6587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:32.742358 6587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:32.742539 6587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:32.742779 6587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:32.743283 6643 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260928 14:41:32.745357 6643 ts_tablet_manager.cc:1852] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260928 14:41:32.745788 6643 ts_tablet_manager.cc:1905] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:32.748656 6643 ts_tablet_manager.cc:1918] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:41:32.748771 6643 log.cc:1196] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/wal/wals/c30359ab370f4623bb6e59ec9af1137d
I20260928 14:41:32.749090 6643 ts_tablet_manager.cc:1939] T c30359ab370f4623bb6e59ec9af1137d P 22e98f4d9dd9467c92809fdd2c8eaf79: Deleting consensus metadata
I20260928 14:41:32.749315 6587 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260928 14:41:32.749379 6587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260928 14:41:32.749428 6587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:32.749454 6587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:32.757085 6587 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:38419
I20260928 14:41:32.757174 6750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:38419 every 8 connection(s)
I20260928 14:41:32.757505 6587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:41:32.761055 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 6587
I20260928 14:41:32.761392 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:41107
--local_ip_for_outbound_sockets=127.22.13.66
--tserver_master_addrs=127.22.13.126:41223
--webserver_port=37605
--webserver_interface=127.22.13.66
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:32.822623 6753 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:41:32.758360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:53223)
0928 14:41:32.758555 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:32.758594 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:32.759105 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:32.822386 (+ 63281us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
W20260928 14:41:32.822845 6751 heartbeater.cc:644] Failed to heartbeat to 127.22.13.126:41223 (0 consecutive failures): Network error: Failed to ping master at 127.22.13.126:41223: Client connection negotiation failed: client connection to 127.22.13.126:41223: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 14:41:32.823237 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:32.823108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:33315)
0928 14:41:32.823172 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:32.823217 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260928 14:41:32.823541 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:32.823441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:44629)
0928 14:41:32.823487 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:32.823523 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 14:41:32.823632 6751 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:41:32.848130 6754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:32.848394 6754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:32.848439 6754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:32.849702 6754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:32.849779 6754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:32.849835 6754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:41:32.851456 6754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:41107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=37605
--tserver_master_addrs=127.22.13.126:41223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:32.851751 6754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:32.852002 6754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:32.857196 6760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:32.857200 6759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:32.857226 6762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:32.857646 6754 server_base.cc:1034] running on GCE node
I20260928 14:41:32.857889 6754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:32.858173 6754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:32.859332 6754 hybrid_clock.cc:648] HybridClock initialized: now 1790606492859307 us; error 38 us; skew 500 ppm
Sep 28 14:41:32 dist-test-slave-wwwb krb5kdc[5449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606492, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:32.861871 6754 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:41:32.862377 6754 webserver.cc:460] Webserver started at http://127.22.13.66:37605/ using document root <none> and password file <none>
I20260928 14:41:32.862612 6754 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:32.862660 6754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:32.864075 6754 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:32.864858 6769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:32.865025 6754 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:32.865116 6754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "ee9c8b5cc3ac444ab1551d1634c47e82"
format_stamp: "Formatted at 2026-09-28 14:40:51 on dist-test-slave-wwwb"
server_key: "dbc961026bc0aeac41e9429e2a575cc8"
server_key_iv: "6ecadece95371e6d67bfb086695bc318"
server_key_version: "encryptionkey@0"
I20260928 14:41:32.865477 6754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:32.874651 6754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:32.876600 6754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:32.876734 6754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:32.876958 6754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:32.877410 6776 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260928 14:41:32.878887 6776 ts_tablet_manager.cc:1852] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260928 14:41:32.879163 6776 ts_tablet_manager.cc:1905] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:32.881389 6776 ts_tablet_manager.cc:1918] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:41:32.881466 6776 log.cc:1196] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/c30359ab370f4623bb6e59ec9af1137d
I20260928 14:41:32.881748 6776 ts_tablet_manager.cc:1939] T c30359ab370f4623bb6e59ec9af1137d P ee9c8b5cc3ac444ab1551d1634c47e82: Deleting consensus metadata
I20260928 14:41:32.882042 6754 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260928 14:41:32.882103 6754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260928 14:41:32.882136 6754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:32.882150 6754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:32.888760 6754 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:41107
I20260928 14:41:32.888875 6883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:41107 every 8 connection(s)
I20260928 14:41:32.889243 6754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:41:32.890440 6886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:32.890065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:37851)
0928 14:41:32.890228 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:32.890300 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
W20260928 14:41:32.890702 6884 heartbeater.cc:644] Failed to heartbeat to 127.22.13.126:41223 (0 consecutive failures): Network error: Failed to ping master at 127.22.13.126:41223: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
I20260928 14:41:32.891036 6886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:32.890927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:37659)
0928 14:41:32.890989 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:32.891016 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 14:41:32.891278 6886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:32.891229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:50885)
0928 14:41:32.891249 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:32.891267 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260928 14:41:32.891376 6884 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 14:41:32.897437 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 6754
I20260928 14:41:32.897773 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:37733
--local_ip_for_outbound_sockets=127.22.13.67
--tserver_master_addrs=127.22.13.126:41223
--webserver_port=33409
--webserver_interface=127.22.13.67
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:32.981664 6887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:32.981905 6887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:32.981961 6887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:32.983381 6887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:32.983438 6887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:32.983465 6887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:41:32.985133 6887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:37733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=33409
--tserver_master_addrs=127.22.13.126:41223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:32.985435 6887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:32.985720 6887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:32.991072 6892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:32.991122 6893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:32.991072 6895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:32.991284 6887 server_base.cc:1034] running on GCE node
I20260928 14:41:32.991545 6887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:32.991829 6887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:32.993081 6887 hybrid_clock.cc:648] HybridClock initialized: now 1790606492993069 us; error 29 us; skew 500 ppm
Sep 28 14:41:32 dist-test-slave-wwwb krb5kdc[5449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606492, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:32.995851 6887 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:41:32.996377 6887 webserver.cc:460] Webserver started at http://127.22.13.67:33409/ using document root <none> and password file <none>
I20260928 14:41:32.996640 6887 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:32.996716 6887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:32.998246 6887 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:32.998919 6902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:32.999176 6887 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:32.999280 6887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "a0ed8637b495448fbf2edae1c8aa4161"
format_stamp: "Formatted at 2026-09-28 14:40:51 on dist-test-slave-wwwb"
server_key: "23972ce1adb883570ab5bd70f174a4bf"
server_key_iv: "bd1ce638133273522b782978b551c787"
server_key_version: "encryptionkey@0"
I20260928 14:41:32.999657 6887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:33.011600 6887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:33.014307 6887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:33.014487 6887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:33.014744 6887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:33.015077 6887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:33.015133 6887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:33.015172 6887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:33.015201 6887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:33.021003 6887 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:37733
I20260928 14:41:33.021051 7015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:37733 every 8 connection(s)
I20260928 14:41:33.021404 6887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:41:33.022518 7018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:33.022094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:60817)
0928 14:41:33.022272 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:33.022346 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":75,"threads_started":1}
W20260928 14:41:33.022903 7016 heartbeater.cc:644] Failed to heartbeat to 127.22.13.126:41223 (0 consecutive failures): Network error: Failed to ping master at 127.22.13.126:41223: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
I20260928 14:41:33.023164 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 6887
I20260928 14:41:33.023231 7018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:33.023108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:57443)
0928 14:41:33.023163 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:33.023193 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260928 14:41:33.023516 7018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:33.023419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:40803)
0928 14:41:33.023468 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:33.023497 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 14:41:33.023659 7016 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 14:41:33.824246 7020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:33.823990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:58291)
0928 14:41:33.824148 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:33.824184 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":64,"threads_started":1}
I20260928 14:41:33.891916 7021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:33.891709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:51205)
0928 14:41:33.891841 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:33.891880 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260928 14:41:34.024180 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:34.023976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:57671)
0928 14:41:34.024099 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:34.024129 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":75,"threads_started":1}
I20260928 14:41:34.825188 7023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:34.824910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:58931)
0928 14:41:34.825086 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:34.825136 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":72,"threads_started":1}
I20260928 14:41:34.892701 7024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:34.892486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:34549)
0928 14:41:34.892618 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:34.892658 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260928 14:41:35.024922 7025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:35.024721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:54203)
0928 14:41:35.024845 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:35.024877 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260928 14:41:35.826005 7026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:35.825756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:51217)
0928 14:41:35.825888 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:35.825929 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260928 14:41:35.893522 7027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:35.893214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:49779)
0928 14:41:35.893430 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:35.893476 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":61,"threads_started":1}
I20260928 14:41:36.025884 7028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:36.025640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:60527)
0928 14:41:36.025791 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:36.025836 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
I20260928 14:41:36.826673 7029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:36.826440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:42837)
0928 14:41:36.826588 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:36.826624 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":65,"threads_started":1}
I20260928 14:41:36.894308 7030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:36.894097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:59023)
0928 14:41:36.894221 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:36.894263 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260928 14:41:37.026654 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:37.026418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:45063)
0928 14:41:37.026561 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:37.026604 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260928 14:41:37.827296 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:37.827096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:36037)
0928 14:41:37.827214 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:37.827247 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
I20260928 14:41:37.895079 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:37.894805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:33839)
0928 14:41:37.894957 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:37.895019 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
I20260928 14:41:38.027441 7034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:38.027208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:46335)
0928 14:41:38.027350 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:38.027394 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260928 14:41:38.828040 7035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:38.827795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:57669)
0928 14:41:38.827944 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:38.827985 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260928 14:41:38.896039 7036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:38.895688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:43737)
0928 14:41:38.895942 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:38.895991 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":63,"threads_started":1}
I20260928 14:41:39.028247 7037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:39.027912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:51423)
0928 14:41:39.028155 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:39.028198 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":62,"threads_started":1}
I20260928 14:41:39.828886 7038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:39.828631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:51479)
0928 14:41:39.828782 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:39.828838 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":69,"threads_started":1}
I20260928 14:41:39.896775 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:39.896531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:57333)
0928 14:41:39.896698 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:39.896728 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260928 14:41:40.029055 7040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:40.028773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:52637)
0928 14:41:40.028920 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:40.028992 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
I20260928 14:41:40.829833 7041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:40.829570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:40317)
0928 14:41:40.829722 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:40.829775 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260928 14:41:40.897541 7042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:40.897331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:55287)
0928 14:41:40.897458 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:40.897491 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
I20260928 14:41:41.029862 7043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:41.029614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:38733)
0928 14:41:41.029767 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:41.029815 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":63,"threads_started":1}
I20260928 14:41:41.830632 7044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:41.830399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:55491)
0928 14:41:41.830547 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:41.830583 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
I20260928 14:41:41.898526 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:41.898263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:51339)
0928 14:41:41.898432 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:41.898477 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260928 14:41:42.030532 7046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:42.030265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:54267)
0928 14:41:42.030451 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:42.030485 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":131,"threads_started":1}
I20260928 14:41:42.831451 7047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:42.831217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:44541)
0928 14:41:42.831359 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:42.831405 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":61,"threads_started":1}
I20260928 14:41:42.899314 7048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:42.899057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:58941)
0928 14:41:42.899217 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:42.899265 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":77,"threads_started":1}
I20260928 14:41:43.031188 7049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:43.030978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:38259)
0928 14:41:43.031105 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:43.031141 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
I20260928 14:41:43.832267 7050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:43.832035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:40465)
0928 14:41:43.832171 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:43.832217 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260928 14:41:43.900247 7051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:43.900013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:36807)
0928 14:41:43.900154 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:43.900200 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260928 14:41:44.031925 7052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:44.031696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:52745)
0928 14:41:44.031832 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:44.031877 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260928 14:41:44.833137 7053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:44.832899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:54333)
0928 14:41:44.833046 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:44.833088 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260928 14:41:44.901167 7054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:44.900886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:39187)
0928 14:41:44.901042 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:44.901088 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":62,"threads_started":1}
I20260928 14:41:45.032680 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:45.032424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:44937)
0928 14:41:45.032585 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:45.032627 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":89,"threads_started":1}
I20260928 14:41:45.834038 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:45.833744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:47499)
0928 14:41:45.833898 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:45.833966 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
I20260928 14:41:45.901868 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:45.901646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:45119)
0928 14:41:45.901784 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:45.901821 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260928 14:41:46.033494 7058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:46.033223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:50637)
0928 14:41:46.033397 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:46.033445 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":81,"threads_started":1}
I20260928 14:41:46.834807 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:46.834530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:44307)
0928 14:41:46.834709 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:46.834758 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":66,"threads_started":1}
I20260928 14:41:46.902732 7060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:46.902479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:33139)
0928 14:41:46.902634 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:46.902671 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":60,"threads_started":1}
I20260928 14:41:47.034461 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:47.034200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.67:54551)
0928 14:41:47.034347 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:47.034401 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260928 14:41:47.835659 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:47.835358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.65:53893)
0928 14:41:47.835567 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:47.835612 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":59,"threads_started":1}
I20260928 14:41:47.903530 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:47.903287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:41223 (local address 127.22.13.66:46083)
0928 14:41:47.903442 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:47.903485 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.126:41223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":78,"threads_started":1}
I20260928 14:41:48.025180 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 6587
I20260928 14:41:48.030086 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 6754
I20260928 14:41:48.034550 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 6887
2026-09-28T14:41:48Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58479 ms)
[----------] 30 tests from SecurityITest (237576 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:41:48 dist-test-slave-wwwb krb5kdc[7066](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:41:48 dist-test-slave-wwwb krb5kdc[7066](info): set up 2 sockets
Sep 28 14:41:48 dist-test-slave-wwwb krb5kdc[7066](info): commencing operation
krb5kdc: starting...
W20260928 14:41:49.069916 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:41:49 dist-test-slave-wwwb krb5kdc[7066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:41:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:41:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:41:49.189227 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:36509
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:36509
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:49.274482 7079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:49.274717 7079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:49.274744 7079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:49.275936 7079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:41:49.275976 7079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:49.275990 7079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:49.276003 7079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:41:49.276015 7079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:41:49.277519 7079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:36509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:36509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:49.277751 7079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:49.278033 7079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:49.283466 7087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:49.283463 7085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:49.283504 7079 server_base.cc:1034] running on GCE node
W20260928 14:41:49.283466 7084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:49.283960 7079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:49.284296 7079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:49.285482 7079 hybrid_clock.cc:648] HybridClock initialized: now 1790606509285477 us; error 38 us; skew 500 ppm
Sep 28 14:41:49 dist-test-slave-wwwb krb5kdc[7066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606509, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:49.288127 7079 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:41:49.288651 7079 webserver.cc:460] Webserver started at http://127.22.13.126:34603/ using document root <none> and password file <none>
I20260928 14:41:49.288894 7079 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:49.288968 7079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:49.289139 7079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:49.290374 7079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "0828e670576d4147ba42ee23758cfb01"
format_stamp: "Formatted at 2026-09-28 14:41:49 on dist-test-slave-wwwb"
server_key: "f34ea5142437df4f7f440361b84d9edc"
server_key_iv: "67e60e3b613d6acb7c6c3d1aef2fbc61"
server_key_version: "encryptionkey@0"
I20260928 14:41:49.290757 7079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "0828e670576d4147ba42ee23758cfb01"
format_stamp: "Formatted at 2026-09-28 14:41:49 on dist-test-slave-wwwb"
server_key: "f34ea5142437df4f7f440361b84d9edc"
server_key_iv: "67e60e3b613d6acb7c6c3d1aef2fbc61"
server_key_version: "encryptionkey@0"
I20260928 14:41:49.292248 7079 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:49.293401 7094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:49.293691 7079 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:49.293793 7079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "0828e670576d4147ba42ee23758cfb01"
format_stamp: "Formatted at 2026-09-28 14:41:49 on dist-test-slave-wwwb"
server_key: "f34ea5142437df4f7f440361b84d9edc"
server_key_iv: "67e60e3b613d6acb7c6c3d1aef2fbc61"
server_key_version: "encryptionkey@0"
I20260928 14:41:49.293900 7079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:49.333528 7079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:49.336014 7079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:49.336131 7079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:49.340191 7079 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:36509
I20260928 14:41:49.340227 7146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:36509 every 8 connection(s)
I20260928 14:41:49.340619 7079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:41:49.341593 7147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:49.343981 7147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01: Bootstrap starting.
I20260928 14:41:49.344579 7147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:49.344813 7147 log.cc:824] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:49.345047 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 7079
I20260928 14:41:49.345180 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:41:49.345399 22581 external_mini_cluster.cc:1428] Setting key d9648f3e0e1df565556e294b9267b4f6
I20260928 14:41:49.345796 7147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01: No bootstrap required, opened a new log
I20260928 14:41:49.348222 7147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0828e670576d4147ba42ee23758cfb01" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36509 } }
I20260928 14:41:49.348420 7147 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:49.348455 7147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0828e670576d4147ba42ee23758cfb01, State: Initialized, Role: FOLLOWER
I20260928 14:41:49.348623 7147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0828e670576d4147ba42ee23758cfb01" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36509 } }
I20260928 14:41:49.348721 7147 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:41:49.348752 7147 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:41:49.348790 7147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:49.349556 7147 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0828e670576d4147ba42ee23758cfb01" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36509 } }
I20260928 14:41:49.349673 7147 leader_election.cc:302] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0828e670576d4147ba42ee23758cfb01; no voters:
I20260928 14:41:49.349834 7147 leader_election.cc:288] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:41:49.349912 7152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:41:49.350121 7147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:41:49.350157 7152 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [term 1 LEADER]: Becoming Leader. State: Replica: 0828e670576d4147ba42ee23758cfb01, State: Running, Role: LEADER
I20260928 14:41:49.350313 7152 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0828e670576d4147ba42ee23758cfb01" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36509 } }
I20260928 14:41:49.350720 7153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0828e670576d4147ba42ee23758cfb01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0828e670576d4147ba42ee23758cfb01" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36509 } } }
I20260928 14:41:49.350809 7153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:49.351001 7154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0828e670576d4147ba42ee23758cfb01. Latest consensus state: current_term: 1 leader_uuid: "0828e670576d4147ba42ee23758cfb01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0828e670576d4147ba42ee23758cfb01" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 36509 } } }
I20260928 14:41:49.351119 7154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:49.351640 7161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:41:49.352164 7161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:41:49.354131 7161 catalog_manager.cc:1347] Generated new cluster ID: ccd52f5524534166961362a55c612d83
I20260928 14:41:49.354184 7161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:41:49 dist-test-slave-wwwb krb5kdc[7066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:41:49.359578 7151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.346039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40840 (local address 127.22.13.126:36509)
0928 14:41:49.346249 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:49.346258 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:49.346293 (+ 35us) server_negotiation.cc:407] Connection header received
0928 14:41:49.347372 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:49.347382 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:49.347453 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:49.347621 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:49.348269 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.350040 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.351318 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.351989 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.352139 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.355136 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:49.355141 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:49.355141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:49.355155 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:49.357282 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:49.357738 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:49.357742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:49.357743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:49.357870 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:49.358125 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:49.358126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:49.358126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:49.358486 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:49.359102 (+ 616us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:49.359186 (+ 84us) server_negotiation.cc:299] Negotiation successful
0928 14:41:49.359344 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
I20260928 14:41:49.378198 7161 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:41:49.378762 7161 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:41:49.383086 7161 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01: Generated new TSK 0
I20260928 14:41:49.383319 7161 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:41:49.413110 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36509
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:49.502022 7173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:49.502406 7173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:49.502452 7173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:49.504402 7173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:49.504475 7173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:49.504525 7173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:41:49.506752 7173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:49.507004 7173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:49.507269 7173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:49.512591 7179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:49.512633 7181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:49.512598 7178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:49.512694 7173 server_base.cc:1034] running on GCE node
I20260928 14:41:49.513067 7173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:49.513376 7173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:49.514545 7173 hybrid_clock.cc:648] HybridClock initialized: now 1790606509514531 us; error 48 us; skew 500 ppm
Sep 28 14:41:49 dist-test-slave-wwwb krb5kdc[7066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606509, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:49.517181 7173 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:41:49.517696 7173 webserver.cc:460] Webserver started at http://127.22.13.65:35251/ using document root <none> and password file <none>
I20260928 14:41:49.518000 7173 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:49.518098 7173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:49.518241 7173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:49.519356 7173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "762919e5a8de40a7b27adaad6902b3d4"
format_stamp: "Formatted at 2026-09-28 14:41:49 on dist-test-slave-wwwb"
server_key: "a962c547b7be153c719b35088b2f940f"
server_key_iv: "2fa4496bccb42d2881ada7af4f2327ff"
server_key_version: "encryptionkey@0"
I20260928 14:41:49.519744 7173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "762919e5a8de40a7b27adaad6902b3d4"
format_stamp: "Formatted at 2026-09-28 14:41:49 on dist-test-slave-wwwb"
server_key: "a962c547b7be153c719b35088b2f940f"
server_key_iv: "2fa4496bccb42d2881ada7af4f2327ff"
server_key_version: "encryptionkey@0"
I20260928 14:41:49.521144 7173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:41:49.521908 7188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:49.522127 7173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:41:49.522234 7173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "762919e5a8de40a7b27adaad6902b3d4"
format_stamp: "Formatted at 2026-09-28 14:41:49 on dist-test-slave-wwwb"
server_key: "a962c547b7be153c719b35088b2f940f"
server_key_iv: "2fa4496bccb42d2881ada7af4f2327ff"
server_key_version: "encryptionkey@0"
I20260928 14:41:49.522324 7173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:49.538944 7173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:49.540884 7173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:49.541072 7173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:49.541322 7173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:49.541659 7173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:49.541713 7173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:49.541749 7173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:49.541792 7173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:49.547880 7173 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:38703
I20260928 14:41:49.547938 7301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:38703 every 8 connection(s)
I20260928 14:41:49.548316 7173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:41:49.548611 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 7173
I20260928 14:41:49.548729 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:41:49.549073 22581 external_mini_cluster.cc:1428] Setting key 8348ef6d9d943f165bb11f22a105be25
Sep 28 14:41:49 dist-test-slave-wwwb krb5kdc[7066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606509, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
I20260928 14:41:49.560072 7151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.549570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:53733 (local address 127.22.13.126:36509)
0928 14:41:49.549618 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:49.549620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:49.549653 (+ 33us) server_negotiation.cc:407] Connection header received
0928 14:41:49.550440 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:49.550441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:49.550454 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:49.550524 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:49.551769 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.552594 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.554277 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.554505 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.554564 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.557176 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:49.557177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:49.557178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:49.557185 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:49.558733 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:49.559150 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:49.559151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:49.559151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:49.559202 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:49.559482 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:49.559483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:49.559483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:49.559603 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:49.559660 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:49.559918 (+ 258us) server_negotiation.cc:299] Negotiation successful
0928 14:41:49.559999 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:41:49.560269 7304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.549276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36509 (local address 127.22.13.65:53733)
0928 14:41:49.549525 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:49.549579 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:49.550326 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:49.550588 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:49.550591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:49.551426 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:49.551662 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:49.551669 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.552716 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.552718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:49.554191 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:49.554195 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.554605 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.554607 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:49.554773 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.555390 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:41:49.555399 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:49.557057 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:49.558820 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:49.558828 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:49.558830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:49.559079 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:49.559294 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:49.559295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:49.559296 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:49.559419 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:49.559665 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:49.559670 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:49.559802 (+ 132us) client_negotiation.cc:769] Sending connection context
0928 14:41:49.559890 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 14:41:49.560037 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":108,"threads_started":1}
I20260928 14:41:49.560868 7302 heartbeater.cc:347] Connected to a master server at 127.22.13.126:36509
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
I20260928 14:41:49.560973 7302 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:49.561184 7302 heartbeater.cc:511] Master 127.22.13.126:36509 requested a full tablet report, sending...
I20260928 14:41:49.561810 7111 ts_manager.cc:194] Registered new tserver with Master: 762919e5a8de40a7b27adaad6902b3d4 (127.22.13.65:38703)
I20260928 14:41:49.562541 7111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:53733
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:41:49.576622 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36509
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:49.658797 7307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:49.659034 7307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:49.659060 7307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:49.660216 7307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:49.660259 7307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:49.660286 7307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:41:49.661724 7307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:49.661926 7307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:49.662220 7307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:49.667330 7315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:49.667332 7312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:49.667574 7307 server_base.cc:1034] running on GCE node
W20260928 14:41:49.667330 7313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:49.667853 7307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:49.668128 7307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:49.669275 7307 hybrid_clock.cc:648] HybridClock initialized: now 1790606509669256 us; error 39 us; skew 500 ppm
Sep 28 14:41:49 dist-test-slave-wwwb krb5kdc[7066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606509, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:49.671871 7307 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:41:49.672338 7307 webserver.cc:460] Webserver started at http://127.22.13.66:39663/ using document root <none> and password file <none>
I20260928 14:41:49.672528 7307 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:49.672569 7307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:49.672659 7307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:49.673694 7307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "ab6d98a48d5b4b73999537a216e4591c"
format_stamp: "Formatted at 2026-09-28 14:41:49 on dist-test-slave-wwwb"
server_key: "8bbcca6be725226ecec5cc1bd09ee580"
server_key_iv: "99b1903364e912dff42f33042f1c36cb"
server_key_version: "encryptionkey@0"
I20260928 14:41:49.674062 7307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "ab6d98a48d5b4b73999537a216e4591c"
format_stamp: "Formatted at 2026-09-28 14:41:49 on dist-test-slave-wwwb"
server_key: "8bbcca6be725226ecec5cc1bd09ee580"
server_key_iv: "99b1903364e912dff42f33042f1c36cb"
server_key_version: "encryptionkey@0"
I20260928 14:41:49.675318 7307 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:49.676008 7322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:49.676191 7307 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:49.676276 7307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "ab6d98a48d5b4b73999537a216e4591c"
format_stamp: "Formatted at 2026-09-28 14:41:49 on dist-test-slave-wwwb"
server_key: "8bbcca6be725226ecec5cc1bd09ee580"
server_key_iv: "99b1903364e912dff42f33042f1c36cb"
server_key_version: "encryptionkey@0"
I20260928 14:41:49.676334 7307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:49.692798 7307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:49.695060 7307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:49.695181 7307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:49.695374 7307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:49.695711 7307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:49.695745 7307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:49.695770 7307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:49.695811 7307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:49.702080 7307 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:45613
I20260928 14:41:49.702198 7435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:45613 every 8 connection(s)
I20260928 14:41:49.702507 7307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
Sep 28 14:41:49 dist-test-slave-wwwb krb5kdc[7066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606509, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:41:49.712013 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 7307
I20260928 14:41:49.712148 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:41:49.712378 22581 external_mini_cluster.cc:1428] Setting key a196e041cd0f0844e4efe631fab4cfaa
I20260928 14:41:49.712575 7151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.703375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:42975 (local address 127.22.13.126:36509)
0928 14:41:49.703462 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:49.703465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:49.703728 (+ 263us) server_negotiation.cc:407] Connection header received
0928 14:41:49.704216 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:49.704217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:49.704230 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:49.704312 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:49.705271 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.706100 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.707104 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.707464 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.707538 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.709821 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:49.709822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:49.709822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:49.709830 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:49.711087 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:49.711471 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:49.711472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:49.711472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:49.711522 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:49.711738 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:49.711739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:49.711739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:49.711842 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:49.711892 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:49.712303 (+ 411us) server_negotiation.cc:299] Negotiation successful
0928 14:41:49.712373 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 14:41:49.712743 7438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.703353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36509 (local address 127.22.13.66:42975)
0928 14:41:49.703587 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:49.703626 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:49.704115 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:49.704367 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:49.704371 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:49.704962 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:49.705173 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:49.705178 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.706200 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.706201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:49.707018 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:49.707021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.707562 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.707562 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:49.707669 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.708267 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:41:49.708276 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:49.709724 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:49.711162 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:49.711169 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:49.711171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:49.711403 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:49.711582 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:49.711583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:49.711583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:49.711684 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:49.711905 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:49.711911 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:49.712104 (+ 193us) client_negotiation.cc:769] Sending connection context
0928 14:41:49.712271 (+ 167us) client_negotiation.cc:240] Negotiation successful
0928 14:41:49.712464 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":143,"threads_started":1}
I20260928 14:41:49.713304 7436 heartbeater.cc:347] Connected to a master server at 127.22.13.126:36509
I20260928 14:41:49.713438 7436 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:49.713646 7436 heartbeater.cc:511] Master 127.22.13.126:36509 requested a full tablet report, sending...
I20260928 14:41:49.714039 7111 ts_manager.cc:194] Registered new tserver with Master: ab6d98a48d5b4b73999537a216e4591c (127.22.13.66:45613)
I20260928 14:41:49.714421 7111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:42975
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:41:49.736788 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36509
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:49.820986 7441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:49.821322 7441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:49.821352 7441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:49.822692 7441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:49.822744 7441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:49.822786 7441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:41:49.824311 7441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:36509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:49.824612 7441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:49.824826 7441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:49.830022 7446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:49.830017 7449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:49.830111 7441 server_base.cc:1034] running on GCE node
W20260928 14:41:49.830080 7447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:49.830554 7441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:49.830785 7441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:49.831924 7441 hybrid_clock.cc:648] HybridClock initialized: now 1790606509831907 us; error 37 us; skew 500 ppm
Sep 28 14:41:49 dist-test-slave-wwwb krb5kdc[7066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606509, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:49.834499 7441 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:41:49.834949 7441 webserver.cc:460] Webserver started at http://127.22.13.67:38983/ using document root <none> and password file <none>
I20260928 14:41:49.835188 7441 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:49.835234 7441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:49.835400 7441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:49.836457 7441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "79f2eba8db09404f982cdbe38c4637f1"
format_stamp: "Formatted at 2026-09-28 14:41:49 on dist-test-slave-wwwb"
server_key: "a4f6e2fc0e514e9b404d7ae8d7223b5c"
server_key_iv: "36addbcb2666871533bec1024024a46b"
server_key_version: "encryptionkey@0"
I20260928 14:41:49.836822 7441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "79f2eba8db09404f982cdbe38c4637f1"
format_stamp: "Formatted at 2026-09-28 14:41:49 on dist-test-slave-wwwb"
server_key: "a4f6e2fc0e514e9b404d7ae8d7223b5c"
server_key_iv: "36addbcb2666871533bec1024024a46b"
server_key_version: "encryptionkey@0"
I20260928 14:41:49.838243 7441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:41:49.839005 7456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:49.839178 7441 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:49.839287 7441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "79f2eba8db09404f982cdbe38c4637f1"
format_stamp: "Formatted at 2026-09-28 14:41:49 on dist-test-slave-wwwb"
server_key: "a4f6e2fc0e514e9b404d7ae8d7223b5c"
server_key_iv: "36addbcb2666871533bec1024024a46b"
server_key_version: "encryptionkey@0"
I20260928 14:41:49.839368 7441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:49.853905 7441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:49.855973 7441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:49.856149 7441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:49.856392 7441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:49.856712 7441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:49.856771 7441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:49.856815 7441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:49.856848 7441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:49.863380 7441 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:44149
I20260928 14:41:49.863447 7569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:44149 every 8 connection(s)
I20260928 14:41:49.863848 7441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
Sep 28 14:41:49 dist-test-slave-wwwb krb5kdc[7066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606509, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:41:49.871932 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 7441
I20260928 14:41:49.872048 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:41:49.872264 22581 external_mini_cluster.cc:1428] Setting key 8edcc8d6247b64b16a6750c2fd081176
I20260928 14:41:49.873193 7151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.864578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:34557 (local address 127.22.13.126:36509)
0928 14:41:49.864623 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:49.864625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:49.864838 (+ 213us) server_negotiation.cc:407] Connection header received
0928 14:41:49.865296 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:49.865297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:49.865305 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:49.865367 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:49.866446 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.866948 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.867924 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.868109 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.868160 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.870334 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:49.870335 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:49.870336 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:49.870342 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:49.871556 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:49.871946 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:49.871947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:49.871948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:49.872014 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:49.872356 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:49.872358 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:49.872358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:49.872519 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:49.872596 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:49.872971 (+ 375us) server_negotiation.cc:299] Negotiation successful
0928 14:41:49.873072 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:41:49.873407 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.864529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:36509 (local address 127.22.13.67:34557)
0928 14:41:49.864694 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:49.864737 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:49.865216 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:49.865462 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:49.865466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:49.866147 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:49.866361 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:49.866366 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.867034 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.867034 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:49.867847 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:49.867849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.868181 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.868182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:49.868283 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.868835 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:41:49.868843 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:49.870247 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:49.871633 (+ 1386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:49.871639 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:49.871641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:49.871862 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:49.872121 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:49.872122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:49.872123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:49.872277 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:49.872626 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:49.872633 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:49.872810 (+ 177us) client_negotiation.cc:769] Sending connection context
0928 14:41:49.872923 (+ 113us) client_negotiation.cc:240] Negotiation successful
0928 14:41:49.873117 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260928 14:41:49.873915 7570 heartbeater.cc:347] Connected to a master server at 127.22.13.126:36509
I20260928 14:41:49.874048 7570 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:49.874210 7570 heartbeater.cc:511] Master 127.22.13.126:36509 requested a full tablet report, sending...
I20260928 14:41:49.874536 7111 ts_manager.cc:194] Registered new tserver with Master: 79f2eba8db09404f982cdbe38c4637f1 (127.22.13.67:44149)
I20260928 14:41:49.874881 7111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:34557
I20260928 14:41:49.884382 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:41:49.891557 7151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.885724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.22.13.126:36509)
0928 14:41:49.885789 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:49.885791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:49.885900 (+ 109us) server_negotiation.cc:407] Connection header received
0928 14:41:49.885988 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:49.885989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:49.886007 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:49.886064 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:49.886614 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.887100 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.887809 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.887993 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.888048 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.889209 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:49.889210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:49.889210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:49.889216 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:49.890563 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:49.890887 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:49.890888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:49.890888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:49.890940 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:49.891106 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:49.891107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:49.891107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:49.891258 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:49.891335 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:49.891425 (+ 90us) server_negotiation.cc:299] Negotiation successful
0928 14:41:49.891488 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:41:49.893191 7111 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:41:49.894819 7111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:41:49.901535 7582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.897195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:44149 (local address 127.0.0.1:58556)
0928 14:41:49.897694 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:49.897703 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:49.897741 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:49.898103 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:49.898104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:49.898106 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:49.898162 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:49.898164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.899459 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.899466 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:49.900454 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:49.900457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.901191 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.901192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:49.901277 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.901291 (+ 14us) client_negotiation.cc:769] Sending connection context
0928 14:41:49.901385 (+ 94us) client_negotiation.cc:240] Negotiation successful
0928 14:41:49.901443 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":84,"threads_started":1}
I20260928 14:41:49.901535 7585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.897456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58556 (local address 127.22.13.67:44149)
0928 14:41:49.897637 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:49.897641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:49.897802 (+ 161us) server_negotiation.cc:407] Connection header received
0928 14:41:49.897848 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:49.897879 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:49.897898 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:49.897998 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:49.898241 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.899352 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.900556 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.901087 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.901190 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.901256 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:49.901401 (+ 145us) server_negotiation.cc:299] Negotiation successful
0928 14:41:49.901447 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":71,"threads_started":1}
I20260928 14:41:49.901774 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.897200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:45613 (local address 127.0.0.1:54956)
0928 14:41:49.897843 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:49.897853 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:49.897962 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:49.898225 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:49.898226 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:49.898228 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:49.898257 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:49.898258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.899178 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.899180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:49.900523 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:49.900524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.901518 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.901519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:49.901580 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.901584 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:41:49.901636 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 14:41:49.901688 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":7,"thread_start_us":83,"threads_started":1}
I20260928 14:41:49.902102 7586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.897607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54956 (local address 127.22.13.66:45613)
0928 14:41:49.897773 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:49.897777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:49.897913 (+ 136us) server_negotiation.cc:407] Connection header received
0928 14:41:49.898061 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:49.898064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:49.898079 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:49.898130 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:49.898342 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.899068 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.900625 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.901353 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.901424 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.901474 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:49.901966 (+ 492us) server_negotiation.cc:299] Negotiation successful
0928 14:41:49.902022 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":70,"threads_started":1}
I20260928 14:41:49.902102 7584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.897370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54436 (local address 127.22.13.65:38703)
0928 14:41:49.897581 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:49.897587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:49.897599 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:41:49.897667 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:49.897670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:49.897686 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:49.897765 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:49.898055 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.899294 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.900625 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.901353 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.901423 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.901473 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:49.901733 (+ 260us) server_negotiation.cc:299] Negotiation successful
0928 14:41:49.902026 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":110,"threads_started":1}
I20260928 14:41:49.902523 7504 tablet_service.cc:1467] Processing CreateTablet for tablet c9ace4ded18943e3b2b70ccb18f5de2d (DEFAULT_TABLE table=test-table [id=1d98d011a07b4b02a92ff84841b7b49f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:49.902750 7370 tablet_service.cc:1467] Processing CreateTablet for tablet c9ace4ded18943e3b2b70ccb18f5de2d (DEFAULT_TABLE table=test-table [id=1d98d011a07b4b02a92ff84841b7b49f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:49.902968 7504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ace4ded18943e3b2b70ccb18f5de2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:49.903046 7370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ace4ded18943e3b2b70ccb18f5de2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:49.906093 7588 tablet_bootstrap.cc:492] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c: Bootstrap starting.
I20260928 14:41:49.906256 7583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.897189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38703 (local address 127.0.0.1:54436)
0928 14:41:49.897446 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:49.897498 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:49.897584 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:49.897857 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:49.897860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:49.897863 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:49.897964 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:49.897966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.899403 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.899404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:49.900520 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:49.900523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.901512 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.901513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:49.901590 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.901595 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:41:49.901640 (+ 45us) client_negotiation.cc:240] Negotiation successful
0928 14:41:49.901689 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":14,"thread_start_us":65,"threads_started":1}
I20260928 14:41:49.906203 7587 tablet_bootstrap.cc:492] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1: Bootstrap starting.
I20260928 14:41:49.906900 7587 tablet_bootstrap.cc:654] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:49.906936 7588 tablet_bootstrap.cc:654] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:49.907074 7236 tablet_service.cc:1467] Processing CreateTablet for tablet c9ace4ded18943e3b2b70ccb18f5de2d (DEFAULT_TABLE table=test-table [id=1d98d011a07b4b02a92ff84841b7b49f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:49.907358 7587 log.cc:824] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:49.907455 7236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ace4ded18943e3b2b70ccb18f5de2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:49.907505 7588 log.cc:824] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:49.908124 7587 tablet_bootstrap.cc:492] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1: No bootstrap required, opened a new log
I20260928 14:41:49.908187 7587 ts_tablet_manager.cc:1397] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:41:49.908198 7588 tablet_bootstrap.cc:492] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c: No bootstrap required, opened a new log
I20260928 14:41:49.908242 7588 ts_tablet_manager.cc:1397] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260928 14:41:49.909883 7588 raft_consensus.cc:348] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab6d98a48d5b4b73999537a216e4591c" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 45613 } } peers { permanent_uuid: "762919e5a8de40a7b27adaad6902b3d4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38703 } } peers { permanent_uuid: "79f2eba8db09404f982cdbe38c4637f1" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 44149 } }
I20260928 14:41:49.910111 7588 raft_consensus.cc:374] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:49.910148 7588 raft_consensus.cc:729] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab6d98a48d5b4b73999537a216e4591c, State: Initialized, Role: FOLLOWER
I20260928 14:41:49.910280 7588 consensus_queue.cc:260] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab6d98a48d5b4b73999537a216e4591c" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 45613 } } peers { permanent_uuid: "762919e5a8de40a7b27adaad6902b3d4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38703 } } peers { permanent_uuid: "79f2eba8db09404f982cdbe38c4637f1" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 44149 } }
I20260928 14:41:49.910307 7587 raft_consensus.cc:348] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab6d98a48d5b4b73999537a216e4591c" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 45613 } } peers { permanent_uuid: "762919e5a8de40a7b27adaad6902b3d4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38703 } } peers { permanent_uuid: "79f2eba8db09404f982cdbe38c4637f1" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 44149 } }
I20260928 14:41:49.910498 7587 raft_consensus.cc:374] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:49.910557 7587 raft_consensus.cc:729] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79f2eba8db09404f982cdbe38c4637f1, State: Initialized, Role: FOLLOWER
I20260928 14:41:49.910597 7588 ts_tablet_manager.cc:1428] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:41:49.910674 7587 consensus_queue.cc:260] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab6d98a48d5b4b73999537a216e4591c" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 45613 } } peers { permanent_uuid: "762919e5a8de40a7b27adaad6902b3d4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38703 } } peers { permanent_uuid: "79f2eba8db09404f982cdbe38c4637f1" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 44149 } }
I20260928 14:41:49.910748 7436 heartbeater.cc:503] Master 127.22.13.126:36509 was elected leader, sending a full tablet report...
I20260928 14:41:49.910956 7587 ts_tablet_manager.cc:1428] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:41:49.911082 7570 heartbeater.cc:503] Master 127.22.13.126:36509 was elected leader, sending a full tablet report...
I20260928 14:41:49.911378 7592 tablet_bootstrap.cc:492] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4: Bootstrap starting.
I20260928 14:41:49.912113 7592 tablet_bootstrap.cc:654] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:49.912629 7592 log.cc:824] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:49.913427 7592 tablet_bootstrap.cc:492] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4: No bootstrap required, opened a new log
I20260928 14:41:49.913488 7592 ts_tablet_manager.cc:1397] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:41:49.914952 7592 raft_consensus.cc:348] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab6d98a48d5b4b73999537a216e4591c" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 45613 } } peers { permanent_uuid: "762919e5a8de40a7b27adaad6902b3d4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38703 } } peers { permanent_uuid: "79f2eba8db09404f982cdbe38c4637f1" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 44149 } }
I20260928 14:41:49.915115 7592 raft_consensus.cc:374] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:49.915138 7592 raft_consensus.cc:729] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 762919e5a8de40a7b27adaad6902b3d4, State: Initialized, Role: FOLLOWER
I20260928 14:41:49.915226 7592 consensus_queue.cc:260] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab6d98a48d5b4b73999537a216e4591c" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 45613 } } peers { permanent_uuid: "762919e5a8de40a7b27adaad6902b3d4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38703 } } peers { permanent_uuid: "79f2eba8db09404f982cdbe38c4637f1" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 44149 } }
I20260928 14:41:49.915479 7592 ts_tablet_manager.cc:1428] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:41:49.915498 7302 heartbeater.cc:503] Master 127.22.13.126:36509 was elected leader, sending a full tablet report...
I20260928 14:41:49.921587 7595 raft_consensus.cc:482] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:41:49.921734 7595 raft_consensus.cc:504] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab6d98a48d5b4b73999537a216e4591c" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 45613 } } peers { permanent_uuid: "762919e5a8de40a7b27adaad6902b3d4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38703 } } peers { permanent_uuid: "79f2eba8db09404f982cdbe38c4637f1" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 44149 } }
I20260928 14:41:49.922202 7595 leader_election.cc:288] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab6d98a48d5b4b73999537a216e4591c (127.22.13.66:45613), 79f2eba8db09404f982cdbe38c4637f1 (127.22.13.67:44149)
I20260928 14:41:49.926215 7585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.922529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:48337 (local address 127.22.13.67:44149)
0928 14:41:49.922572 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:49.922574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:49.922641 (+ 67us) server_negotiation.cc:407] Connection header received
0928 14:41:49.922702 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:49.922703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:49.922710 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:49.922766 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:49.922915 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.923643 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.925178 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.925815 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.925896 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.925906 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:49.926094 (+ 188us) server_negotiation.cc:299] Negotiation successful
0928 14:41:49.926138 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:41:49.926215 7586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.922469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:50511 (local address 127.22.13.66:45613)
0928 14:41:49.922526 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:49.922528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:49.922538 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:41:49.922568 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:49.922569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:49.922583 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:49.922637 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:49.922845 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.923638 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.925178 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.925815 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.925897 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.925906 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:49.926093 (+ 187us) server_negotiation.cc:299] Negotiation successful
0928 14:41:49.926150 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260928 14:41:49.926250 7304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.922351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:45613 (local address 127.22.13.65:50511)
0928 14:41:49.922421 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:49.922435 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:49.922484 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:49.922659 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:49.922660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:49.922662 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:49.922767 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:49.922770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.923741 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.923741 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:49.925077 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:49.925081 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.925903 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.925904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:49.926015 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.926027 (+ 12us) client_negotiation.cc:769] Sending connection context
0928 14:41:49.926079 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 14:41:49.926141 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":64}
I20260928 14:41:49.926608 7390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ace4ded18943e3b2b70ccb18f5de2d" candidate_uuid: "762919e5a8de40a7b27adaad6902b3d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab6d98a48d5b4b73999537a216e4591c" is_pre_election: true
I20260928 14:41:49.926258 7596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.922419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:44149 (local address 127.22.13.65:48337)
0928 14:41:49.922594 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:49.922602 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:49.922651 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:49.922779 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:49.922780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:49.922782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:49.922842 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:49.922843 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.923741 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.923741 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:49.925077 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:49.925081 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.925903 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.925904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:49.926012 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.926017 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:41:49.926080 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 14:41:49.926140 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":76,"thread_start_us":122,"threads_started":1}
I20260928 14:41:49.926796 7390 raft_consensus.cc:2454] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 762919e5a8de40a7b27adaad6902b3d4 in term 0.
I20260928 14:41:49.927029 7524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ace4ded18943e3b2b70ccb18f5de2d" candidate_uuid: "762919e5a8de40a7b27adaad6902b3d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79f2eba8db09404f982cdbe38c4637f1" is_pre_election: true
I20260928 14:41:49.927042 7190 leader_election.cc:302] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 762919e5a8de40a7b27adaad6902b3d4, ab6d98a48d5b4b73999537a216e4591c; no voters:
I20260928 14:41:49.927145 7524 raft_consensus.cc:2454] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 762919e5a8de40a7b27adaad6902b3d4 in term 0.
I20260928 14:41:49.927197 7595 raft_consensus.cc:2781] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:41:49.927254 7595 raft_consensus.cc:482] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:41:49.927285 7595 raft_consensus.cc:3037] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:49.928038 7595 raft_consensus.cc:504] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab6d98a48d5b4b73999537a216e4591c" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 45613 } } peers { permanent_uuid: "762919e5a8de40a7b27adaad6902b3d4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38703 } } peers { permanent_uuid: "79f2eba8db09404f982cdbe38c4637f1" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 44149 } }
I20260928 14:41:49.928196 7595 leader_election.cc:288] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [CANDIDATE]: Term 1 election: Requested vote from peers ab6d98a48d5b4b73999537a216e4591c (127.22.13.66:45613), 79f2eba8db09404f982cdbe38c4637f1 (127.22.13.67:44149)
I20260928 14:41:49.928337 7524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ace4ded18943e3b2b70ccb18f5de2d" candidate_uuid: "762919e5a8de40a7b27adaad6902b3d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79f2eba8db09404f982cdbe38c4637f1"
I20260928 14:41:49.928362 7390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ace4ded18943e3b2b70ccb18f5de2d" candidate_uuid: "762919e5a8de40a7b27adaad6902b3d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab6d98a48d5b4b73999537a216e4591c"
I20260928 14:41:49.928437 7524 raft_consensus.cc:3037] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:49.928440 7390 raft_consensus.cc:3037] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:49.929215 7390 raft_consensus.cc:2454] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 762919e5a8de40a7b27adaad6902b3d4 in term 1.
I20260928 14:41:49.929323 7524 raft_consensus.cc:2454] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 762919e5a8de40a7b27adaad6902b3d4 in term 1.
I20260928 14:41:49.929389 7190 leader_election.cc:302] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 762919e5a8de40a7b27adaad6902b3d4, ab6d98a48d5b4b73999537a216e4591c; no voters:
I20260928 14:41:49.929486 7595 raft_consensus.cc:2781] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:41:49.929685 7595 raft_consensus.cc:686] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [term 1 LEADER]: Becoming Leader. State: Replica: 762919e5a8de40a7b27adaad6902b3d4, State: Running, Role: LEADER
I20260928 14:41:49.929788 7595 consensus_queue.cc:237] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab6d98a48d5b4b73999537a216e4591c" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 45613 } } peers { permanent_uuid: "762919e5a8de40a7b27adaad6902b3d4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38703 } } peers { permanent_uuid: "79f2eba8db09404f982cdbe38c4637f1" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 44149 } }
I20260928 14:41:49.930730 7110 catalog_manager.cc:5495] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 762919e5a8de40a7b27adaad6902b3d4 (127.22.13.65). New cstate: current_term: 1 leader_uuid: "762919e5a8de40a7b27adaad6902b3d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab6d98a48d5b4b73999537a216e4591c" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "762919e5a8de40a7b27adaad6902b3d4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79f2eba8db09404f982cdbe38c4637f1" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 44149 } health_report { overall_health: UNKNOWN } } }
I20260928 14:41:49.936717 7584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.934526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54442 (local address 127.22.13.65:38703)
0928 14:41:49.934584 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:49.934586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:49.934595 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:41:49.934652 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:49.934653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:49.934670 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:49.934721 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:41:49.934823 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.935287 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.936010 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:49.936190 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:49.936240 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:49.936373 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:41:49.936471 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:41:49.936533 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:49.936570 (+ 37us) server_negotiation.cc:299] Negotiation successful
0928 14:41:49.936588 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 14:41:49.938668 7390 raft_consensus.cc:1261] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c [term 1 FOLLOWER]: Refusing update from remote peer 762919e5a8de40a7b27adaad6902b3d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:41:49.938668 7524 raft_consensus.cc:1261] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1 [term 1 FOLLOWER]: Refusing update from remote peer 762919e5a8de40a7b27adaad6902b3d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:41:49.939039 7597 consensus_queue.cc:1035] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79f2eba8db09404f982cdbe38c4637f1" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 44149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:41:49.939179 7595 consensus_queue.cc:1035] T c9ace4ded18943e3b2b70ccb18f5de2d P 762919e5a8de40a7b27adaad6902b3d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab6d98a48d5b4b73999537a216e4591c" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:41:49.941665 7600 mvcc.cc:205] Tried to move back new op lower bound from 7334324264705257472 to 7334324264673308672. Current Snapshot: MvccSnapshot[applied={T|T < 7334324264705257472}]
I20260928 14:41:49.942003 7601 mvcc.cc:205] Tried to move back new op lower bound from 7334324264705257472 to 7334324264673308672. Current Snapshot: MvccSnapshot[applied={T|T < 7334324264705257472}]
I20260928 14:41:49.942880 7602 mvcc.cc:205] Tried to move back new op lower bound from 7334324264705257472 to 7334324264673308672. Current Snapshot: MvccSnapshot[applied={T|T < 7334324264705257472}]
I20260928 14:41:49.944387 7110 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:41:49.944482 7110 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:41:49.945279 7110 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0828e670576d4147ba42ee23758cfb01: Sending DeleteTablet for 3 replicas of tablet c9ace4ded18943e3b2b70ccb18f5de2d
I20260928 14:41:49.945606 7236 tablet_service.cc:1514] Processing DeleteTablet for tablet c9ace4ded18943e3b2b70ccb18f5de2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:49 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:54436
I20260928 14:41:49.945653 7370 tablet_service.cc:1514] Processing DeleteTablet for tablet c9ace4ded18943e3b2b70ccb18f5de2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:49 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:54956
I20260928 14:41:49.945660 7504 tablet_service.cc:1514] Processing DeleteTablet for tablet c9ace4ded18943e3b2b70ccb18f5de2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:49 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:58556
I20260928 14:41:49.946058 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 7173
I20260928 14:41:49.947140 7612 tablet_replica.cc:330] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1: stopping tablet replica
I20260928 14:41:49.947274 7612 raft_consensus.cc:2229] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:41:49.947417 7612 raft_consensus.cc:2258] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:49.947594 7611 tablet_replica.cc:330] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c: stopping tablet replica
I20260928 14:41:49.947712 7611 raft_consensus.cc:2229] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:41:49.947808 7611 raft_consensus.cc:2258] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:49.948256 7611 ts_tablet_manager.cc:1905] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:49.948256 7612 ts_tablet_manager.cc:1905] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:49.950074 7611 ts_tablet_manager.cc:1918] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:41:49.950074 7612 ts_tablet_manager.cc:1918] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:41:49.950157 7612 log.cc:1196] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/c9ace4ded18943e3b2b70ccb18f5de2d
I20260928 14:41:49.950193 7611 log.cc:1196] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/c9ace4ded18943e3b2b70ccb18f5de2d
I20260928 14:41:49.950464 7612 ts_tablet_manager.cc:1939] T c9ace4ded18943e3b2b70ccb18f5de2d P 79f2eba8db09404f982cdbe38c4637f1: Deleting consensus metadata
I20260928 14:41:49.950484 7611 ts_tablet_manager.cc:1939] T c9ace4ded18943e3b2b70ccb18f5de2d P ab6d98a48d5b4b73999537a216e4591c: Deleting consensus metadata
I20260928 14:41:49.950908 7096 catalog_manager.cc:4841] TS ab6d98a48d5b4b73999537a216e4591c (127.22.13.66:45613): tablet c9ace4ded18943e3b2b70ccb18f5de2d (table test-table [id=1d98d011a07b4b02a92ff84841b7b49f]) successfully deleted
I20260928 14:41:49.951331 7097 catalog_manager.cc:4841] TS 79f2eba8db09404f982cdbe38c4637f1 (127.22.13.67:44149): tablet c9ace4ded18943e3b2b70ccb18f5de2d (table test-table [id=1d98d011a07b4b02a92ff84841b7b49f]) successfully deleted
W20260928 14:41:49.953125 7095 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:38703 (error 108)
I20260928 14:41:49.953356 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 7307
I20260928 14:41:49.954550 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:49.954187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38703 (local address 127.0.0.1:54454)
0928 14:41:49.954441 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:49.954496 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:38703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222}
W20260928 14:41:49.958470 7095 catalog_manager.cc:4584] TS 762919e5a8de40a7b27adaad6902b3d4 (127.22.13.65:38703): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c9ace4ded18943e3b2b70ccb18f5de2d: Network error: Client connection negotiation failed: client connection to 127.22.13.65:38703: connect: Connection refused (error 111)
I20260928 14:41:49.958565 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 7441
I20260928 14:41:49.963244 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 7079
2026-09-28T14:41:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1929 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-28T14:41:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:41:49Z Disabled control of system clock
I20260928 14:41:49.991364 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:32775
--webserver_interface=127.22.13.126
--webserver_port=0
--builtin_ntp_servers=127.22.13.84:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:32775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:41:50.072860 7617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:50.073093 7617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:50.073115 7617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:50.074275 7617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:41:50.074312 7617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:50.074325 7617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:50.074337 7617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:41:50.074349 7617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:41:50.075810 7617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:32775
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:32775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:50.076021 7617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:50.076287 7617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:50.081210 7625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:50.081221 7623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:50.081210 7622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:50.081502 7617 server_base.cc:1034] running on GCE node
I20260928 14:41:50.081770 7617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:50.082120 7617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:50.083267 7617 hybrid_clock.cc:648] HybridClock initialized: now 1790606510083257 us; error 34 us; skew 500 ppm
I20260928 14:41:50.084623 7617 webserver.cc:460] Webserver started at http://127.22.13.126:44817/ using document root <none> and password file <none>
I20260928 14:41:50.084898 7617 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:50.084946 7617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:50.085114 7617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:50.086335 7617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "0f254a255beb4e1087fcc38380c35207"
format_stamp: "Formatted at 2026-09-28 14:41:50 on dist-test-slave-wwwb"
server_key: "c2f724b1c365477121a55bd40c3d4b21"
server_key_iv: "fcdd8b2a193397cee19bc19f51362804"
server_key_version: "encryptionkey@0"
I20260928 14:41:50.086710 7617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "0f254a255beb4e1087fcc38380c35207"
format_stamp: "Formatted at 2026-09-28 14:41:50 on dist-test-slave-wwwb"
server_key: "c2f724b1c365477121a55bd40c3d4b21"
server_key_iv: "fcdd8b2a193397cee19bc19f51362804"
server_key_version: "encryptionkey@0"
I20260928 14:41:50.088083 7617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:41:50.088863 7631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:50.089059 7617 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:50.089171 7617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "0f254a255beb4e1087fcc38380c35207"
format_stamp: "Formatted at 2026-09-28 14:41:50 on dist-test-slave-wwwb"
server_key: "c2f724b1c365477121a55bd40c3d4b21"
server_key_iv: "fcdd8b2a193397cee19bc19f51362804"
server_key_version: "encryptionkey@0"
I20260928 14:41:50.089253 7617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:50.103302 7617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:50.103643 7617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:50.103803 7617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:50.107829 7617 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:32775
I20260928 14:41:50.107870 7683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:32775 every 8 connection(s)
I20260928 14:41:50.108238 7617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:41:50.109228 7684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:50.111626 7684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207: Bootstrap starting.
I20260928 14:41:50.112221 7684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:50.112526 7684 log.cc:824] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:50.113233 7684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207: No bootstrap required, opened a new log
I20260928 14:41:50.114926 7684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f254a255beb4e1087fcc38380c35207" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 32775 } }
I20260928 14:41:50.115074 7684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:50.115098 7684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f254a255beb4e1087fcc38380c35207, State: Initialized, Role: FOLLOWER
I20260928 14:41:50.115272 7684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f254a255beb4e1087fcc38380c35207" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 32775 } }
I20260928 14:41:50.115334 7684 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:41:50.115355 7684 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:41:50.115425 7684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:50.115828 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 7617
I20260928 14:41:50.115947 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:41:50.116189 22581 external_mini_cluster.cc:1428] Setting key e8dd0e9be94f6d5b0b8f71fe2617610b
I20260928 14:41:50.116228 7684 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f254a255beb4e1087fcc38380c35207" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 32775 } }
I20260928 14:41:50.116357 7684 leader_election.cc:302] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f254a255beb4e1087fcc38380c35207; no voters:
I20260928 14:41:50.116591 7684 leader_election.cc:288] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:41:50.116639 7687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:41:50.116781 7687 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [term 1 LEADER]: Becoming Leader. State: Replica: 0f254a255beb4e1087fcc38380c35207, State: Running, Role: LEADER
I20260928 14:41:50.116852 7687 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f254a255beb4e1087fcc38380c35207" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 32775 } }
I20260928 14:41:50.116885 7684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:41:50.117182 7687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f254a255beb4e1087fcc38380c35207" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f254a255beb4e1087fcc38380c35207" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 32775 } } }
I20260928 14:41:50.117257 7687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:50.117381 7687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f254a255beb4e1087fcc38380c35207. Latest consensus state: current_term: 1 leader_uuid: "0f254a255beb4e1087fcc38380c35207" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f254a255beb4e1087fcc38380c35207" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 32775 } } }
I20260928 14:41:50.117448 7687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:50.117712 7694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:41:50.118261 7694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:41:50.119803 7694 catalog_manager.cc:1347] Generated new cluster ID: 535283c9b59e42ba8ca457ea54215863
I20260928 14:41:50.119856 7694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:41:50.123205 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.116778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47018 (local address 127.22.13.126:32775)
0928 14:41:50.117125 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:50.117153 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:50.117193 (+ 40us) server_negotiation.cc:407] Connection header received
0928 14:41:50.117419 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:50.117427 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:50.117588 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:50.118109 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:50.118212 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.119751 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.120894 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.121491 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.121615 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.122448 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:50.122452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:50.122453 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:50.122674 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:50.122721 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:50.122738 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:50.122812 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:50.122885 (+ 73us) server_negotiation.cc:299] Negotiation successful
0928 14:41:50.122976 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":79,"threads_started":1}
I20260928 14:41:50.137409 7694 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:41:50.138207 7694 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:41:50.148576 7694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207: Generated new TSK 0
I20260928 14:41:50.148811 7694 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:41:50.153891 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:32775
--builtin_ntp_servers=127.22.13.84:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:41:50.238193 7708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:50.238503 7708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:50.238541 7708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:50.239795 7708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:50.239871 7708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:50.239915 7708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:41:50.241400 7708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:32775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:50.241703 7708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:50.242079 7708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:50.247723 7713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:50.247846 7716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:50.247730 7714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:50.248173 7708 server_base.cc:1034] running on GCE node
I20260928 14:41:50.248416 7708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:50.248683 7708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:50.249861 7708 hybrid_clock.cc:648] HybridClock initialized: now 1790606510249843 us; error 34 us; skew 500 ppm
I20260928 14:41:50.251176 7708 webserver.cc:460] Webserver started at http://127.22.13.65:45423/ using document root <none> and password file <none>
I20260928 14:41:50.251463 7708 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:50.251536 7708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:50.251683 7708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:50.252801 7708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "b8e52dc508824df8a17f648ec109a8ef"
format_stamp: "Formatted at 2026-09-28 14:41:50 on dist-test-slave-wwwb"
server_key: "fb908dd46432dfd3af797236851b65d2"
server_key_iv: "61636f94cf8513e367ebb1e605e8d656"
server_key_version: "encryptionkey@0"
I20260928 14:41:50.253202 7708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "b8e52dc508824df8a17f648ec109a8ef"
format_stamp: "Formatted at 2026-09-28 14:41:50 on dist-test-slave-wwwb"
server_key: "fb908dd46432dfd3af797236851b65d2"
server_key_iv: "61636f94cf8513e367ebb1e605e8d656"
server_key_version: "encryptionkey@0"
I20260928 14:41:50.254637 7708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:41:50.255380 7722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:50.255565 7708 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:50.255671 7708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "b8e52dc508824df8a17f648ec109a8ef"
format_stamp: "Formatted at 2026-09-28 14:41:50 on dist-test-slave-wwwb"
server_key: "fb908dd46432dfd3af797236851b65d2"
server_key_iv: "61636f94cf8513e367ebb1e605e8d656"
server_key_version: "encryptionkey@0"
I20260928 14:41:50.255775 7708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:50.283891 7708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:50.284289 7708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:50.284474 7708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:50.284759 7708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:50.285120 7708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:50.285180 7708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:50.285215 7708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:50.285257 7708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:50.291271 7708 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:40441
I20260928 14:41:50.291313 7835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:40441 every 8 connection(s)
I20260928 14:41:50.291677 7708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:41:50.297700 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.292464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:44953 (local address 127.22.13.126:32775)
0928 14:41:50.292511 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:50.292513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:50.292777 (+ 264us) server_negotiation.cc:407] Connection header received
0928 14:41:50.293386 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:50.293387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:50.293397 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:50.293461 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:50.294247 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.294748 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.295964 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.296278 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.296355 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.297376 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:50.297378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:50.297378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:50.297383 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:50.297413 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:50.297418 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:50.297445 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:50.297576 (+ 131us) server_negotiation.cc:299] Negotiation successful
0928 14:41:50.297617 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:41:50.297979 7838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.292446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:32775 (local address 127.22.13.65:44953)
0928 14:41:50.292642 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:50.292685 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:50.293294 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:50.293553 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:50.293556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:50.293566 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:50.294163 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:50.294169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.294840 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.294841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:50.295863 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:50.295866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.296377 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.296378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:50.296543 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.297172 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:41:50.297172 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:50.297212 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:41:50.297244 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:50.297487 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:50.297493 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:50.297496 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:50.297565 (+ 69us) client_negotiation.cc:240] Negotiation successful
0928 14:41:50.297675 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260928 14:41:50.298559 7836 heartbeater.cc:347] Connected to a master server at 127.22.13.126:32775
I20260928 14:41:50.298691 7836 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:50.298853 7836 heartbeater.cc:511] Master 127.22.13.126:32775 requested a full tablet report, sending...
I20260928 14:41:50.299302 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 7708
I20260928 14:41:50.299400 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:41:50.299568 22581 external_mini_cluster.cc:1428] Setting key d1baa7fe4e18f5f98553581caf314ff8
I20260928 14:41:50.299557 7648 ts_manager.cc:194] Registered new tserver with Master: b8e52dc508824df8a17f648ec109a8ef (127.22.13.65:40441)
I20260928 14:41:50.300205 7648 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.13.65:44953
I20260928 14:41:50.300700 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:32775
--builtin_ntp_servers=127.22.13.84:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:41:50.383285 7839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:50.383544 7839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:50.383581 7839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:50.384807 7839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:50.384876 7839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:50.384917 7839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:41:50.386430 7839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:32775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:50.386708 7839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:50.387032 7839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:50.392072 7844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:50.392095 7845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:50.392095 7847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:50.392463 7839 server_base.cc:1034] running on GCE node
I20260928 14:41:50.392748 7839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:50.393010 7839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:50.394165 7839 hybrid_clock.cc:648] HybridClock initialized: now 1790606510394147 us; error 36 us; skew 500 ppm
I20260928 14:41:50.395431 7839 webserver.cc:460] Webserver started at http://127.22.13.66:34055/ using document root <none> and password file <none>
I20260928 14:41:50.395686 7839 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:50.395757 7839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:50.395915 7839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:50.397056 7839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "1ff0f9c67e634553958fff89455fc83d"
format_stamp: "Formatted at 2026-09-28 14:41:50 on dist-test-slave-wwwb"
server_key: "0f5096b5abe2bece1e459c466e221d50"
server_key_iv: "07e326024328c5bf6ef1a36dd8a6b2b9"
server_key_version: "encryptionkey@0"
I20260928 14:41:50.397435 7839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "1ff0f9c67e634553958fff89455fc83d"
format_stamp: "Formatted at 2026-09-28 14:41:50 on dist-test-slave-wwwb"
server_key: "0f5096b5abe2bece1e459c466e221d50"
server_key_iv: "07e326024328c5bf6ef1a36dd8a6b2b9"
server_key_version: "encryptionkey@0"
I20260928 14:41:50.398955 7839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:41:50.399705 7853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:50.399904 7839 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:50.400002 7839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "1ff0f9c67e634553958fff89455fc83d"
format_stamp: "Formatted at 2026-09-28 14:41:50 on dist-test-slave-wwwb"
server_key: "0f5096b5abe2bece1e459c466e221d50"
server_key_iv: "07e326024328c5bf6ef1a36dd8a6b2b9"
server_key_version: "encryptionkey@0"
I20260928 14:41:50.400097 7839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:50.420895 7839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:50.421258 7839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:50.421427 7839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:50.421671 7839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:50.422088 7839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:50.422144 7839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:50.422186 7839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:50.422214 7839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:50.428399 7839 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:38449
I20260928 14:41:50.428443 7966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:38449 every 8 connection(s)
I20260928 14:41:50.428822 7839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:41:50.434204 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.429636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:38767 (local address 127.22.13.126:32775)
0928 14:41:50.429706 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:50.429709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:50.429945 (+ 236us) server_negotiation.cc:407] Connection header received
0928 14:41:50.430510 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:50.430511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:50.430521 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:50.430598 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:50.431292 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.431794 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.432791 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.432977 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.433031 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.433885 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:50.433886 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:50.433886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:50.433892 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:50.433906 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:50.433910 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:50.434018 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:50.434108 (+ 90us) server_negotiation.cc:299] Negotiation successful
0928 14:41:50.434137 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:41:50.434407 7969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.429600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:32775 (local address 127.22.13.66:38767)
0928 14:41:50.429796 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:50.429841 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:50.430408 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:50.430665 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:50.430668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:50.430676 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:50.431204 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:50.431208 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.431882 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.431882 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:50.432703 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:50.432705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.433056 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.433057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:50.433172 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.433722 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:41:50.433722 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:50.433751 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:41:50.433795 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:50.434022 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:50.434027 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:50.434030 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:50.434103 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 14:41:50.434192 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260928 14:41:50.434926 7967 heartbeater.cc:347] Connected to a master server at 127.22.13.126:32775
I20260928 14:41:50.435034 7967 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:50.435196 7967 heartbeater.cc:511] Master 127.22.13.126:32775 requested a full tablet report, sending...
I20260928 14:41:50.435541 7648 ts_manager.cc:194] Registered new tserver with Master: 1ff0f9c67e634553958fff89455fc83d (127.22.13.66:38449)
I20260928 14:41:50.435877 7648 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.13.66:38767
I20260928 14:41:50.436156 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 7839
I20260928 14:41:50.436287 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:41:50.436478 22581 external_mini_cluster.cc:1428] Setting key 257abc9f81c894e4346fb66c4408377a
I20260928 14:41:50.437369 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:32775
--builtin_ntp_servers=127.22.13.84:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:41:50.520511 7970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:50.520731 7970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:50.520754 7970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:50.521996 7970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:50.522048 7970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:50.522074 7970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:41:50.523449 7970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:32775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:50.523684 7970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:50.523993 7970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:50.529124 7975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:50.529131 7978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:50.529148 7970 server_base.cc:1034] running on GCE node
W20260928 14:41:50.529131 7976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:50.529541 7970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:50.529764 7970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:50.530906 7970 hybrid_clock.cc:648] HybridClock initialized: now 1790606510530898 us; error 28 us; skew 500 ppm
I20260928 14:41:50.532197 7970 webserver.cc:460] Webserver started at http://127.22.13.67:37709/ using document root <none> and password file <none>
I20260928 14:41:50.532438 7970 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:50.532487 7970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:50.532727 7970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:50.533833 7970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "91f5cbc8797b4b2bb3823be396b6bc3a"
format_stamp: "Formatted at 2026-09-28 14:41:50 on dist-test-slave-wwwb"
server_key: "fc152c77eafb154c64ecf733015be128"
server_key_iv: "f3277f504640992a87aad9756aa430ae"
server_key_version: "encryptionkey@0"
I20260928 14:41:50.534274 7970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "91f5cbc8797b4b2bb3823be396b6bc3a"
format_stamp: "Formatted at 2026-09-28 14:41:50 on dist-test-slave-wwwb"
server_key: "fc152c77eafb154c64ecf733015be128"
server_key_iv: "f3277f504640992a87aad9756aa430ae"
server_key_version: "encryptionkey@0"
I20260928 14:41:50.535594 7970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:50.536286 7984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:50.536453 7970 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:50.536551 7970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "91f5cbc8797b4b2bb3823be396b6bc3a"
format_stamp: "Formatted at 2026-09-28 14:41:50 on dist-test-slave-wwwb"
server_key: "fc152c77eafb154c64ecf733015be128"
server_key_iv: "f3277f504640992a87aad9756aa430ae"
server_key_version: "encryptionkey@0"
I20260928 14:41:50.536638 7970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:50.562134 7970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:50.562529 7970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:50.562705 7970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:50.562948 7970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:50.563298 7970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:50.563364 7970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:50.563405 7970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:50.563446 7970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:50.569191 7970 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:37977
I20260928 14:41:50.569238 8097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:37977 every 8 connection(s)
I20260928 14:41:50.569598 7970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:41:50.572772 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 7970
I20260928 14:41:50.572866 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:41:50.573069 22581 external_mini_cluster.cc:1428] Setting key d63f065dc0d13f664ec6dd192b71cb02
I20260928 14:41:50.575449 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.570437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:59611 (local address 127.22.13.126:32775)
0928 14:41:50.570513 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:50.570516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:50.570815 (+ 299us) server_negotiation.cc:407] Connection header received
0928 14:41:50.571291 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:50.571292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:50.571302 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:50.571357 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:50.572064 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.572569 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.574103 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.574313 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.574370 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.575244 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:50.575245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:50.575245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:50.575249 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:50.575260 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:50.575264 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:50.575301 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:50.575360 (+ 59us) server_negotiation.cc:299] Negotiation successful
0928 14:41:50.575388 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 14:41:50.575698 8100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.570447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:32775 (local address 127.22.13.67:59611)
0928 14:41:50.570668 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:50.570704 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:50.571201 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:50.571436 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:50.571439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:50.571448 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:50.571980 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:50.571984 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.572701 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.572702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:50.574017 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:50.574021 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.574392 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.574393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:50.574513 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.575114 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:41:50.575114 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:50.575144 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:41:50.575170 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:50.575301 (+ 131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:50.575304 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:50.575306 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:41:50.575353 (+ 47us) client_negotiation.cc:240] Negotiation successful
0928 14:41:50.575436 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":89,"threads_started":1}
I20260928 14:41:50.576232 8098 heartbeater.cc:347] Connected to a master server at 127.22.13.126:32775
I20260928 14:41:50.576332 8098 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:50.576483 8098 heartbeater.cc:511] Master 127.22.13.126:32775 requested a full tablet report, sending...
I20260928 14:41:50.576880 7648 ts_manager.cc:194] Registered new tserver with Master: 91f5cbc8797b4b2bb3823be396b6bc3a (127.22.13.67:37977)
I20260928 14:41:50.577206 7648 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.13.67:59611
I20260928 14:41:50.585111 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:41:50.590034 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.586705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47026 (local address 127.22.13.126:32775)
0928 14:41:50.586793 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:50.586796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:50.586894 (+ 98us) server_negotiation.cc:407] Connection header received
0928 14:41:50.586996 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:50.586997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:50.587023 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:50.587075 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:50.587259 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.587778 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.588685 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.588973 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.589052 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.589711 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:50.589712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:50.589713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:50.589719 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:50.589734 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:50.589739 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:50.589808 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:50.589870 (+ 62us) server_negotiation.cc:299] Negotiation successful
0928 14:41:50.589899 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 14:41:50.591777 7648 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:41:50.593601 7648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:41:50.600623 8109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.596055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59490 (local address 127.22.13.66:38449)
0928 14:41:50.596699 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:50.596703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:50.596712 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:41:50.596753 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:50.596755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:50.596783 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:50.596861 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:50.596964 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.598037 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.599307 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.600087 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.600163 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.600260 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:50.600499 (+ 239us) server_negotiation.cc:299] Negotiation successful
0928 14:41:50.600540 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":92,"threads_started":1}
I20260928 14:41:50.600610 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.595888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:38449 (local address 127.0.0.1:59490)
0928 14:41:50.596001 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:50.596041 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:50.596110 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:50.596857 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:50.596857 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:50.596859 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:50.596891 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:50.596893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.598113 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.598114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:50.599204 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:50.599206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.600293 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.600294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:50.600372 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.600381 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 14:41:50.600465 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 14:41:50.600524 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":65,"threads_started":1}
I20260928 14:41:50.601254 8110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.595888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:40441 (local address 127.0.0.1:52772)
0928 14:41:50.596204 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:50.596215 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:50.596269 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:50.596594 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:50.596670 (+ 76us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:50.596672 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:50.596727 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:50.596729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.598921 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.598922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:50.600061 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:50.600065 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.601000 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.601001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:50.601067 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.601072 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:41:50.601132 (+ 60us) client_negotiation.cc:240] Negotiation successful
0928 14:41:50.601178 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":3,"thread_start_us":59,"threads_started":1}
I20260928 14:41:50.601482 8107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.596097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:37977 (local address 127.0.0.1:48826)
0928 14:41:50.596214 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:50.596224 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:50.596261 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:50.596637 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:50.596641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:50.596643 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:50.596727 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:50.596729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.598979 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.598980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:50.599859 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:50.599862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.601083 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.601084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:50.601144 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.601149 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:41:50.601204 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 14:41:50.601250 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":5,"thread_start_us":57,"threads_started":1}
I20260928 14:41:50.601606 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.596295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48826 (local address 127.22.13.67:37977)
0928 14:41:50.596467 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:50.596471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:50.596480 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:41:50.596537 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:50.596539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:50.596554 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:50.596626 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:50.596809 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.597900 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.599956 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.600881 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.600957 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.601045 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:50.601296 (+ 251us) server_negotiation.cc:299] Negotiation successful
0928 14:41:50.601518 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260928 14:41:50.601644 7901 tablet_service.cc:1467] Processing CreateTablet for tablet 8d368db862f44981be7df2464bb2ed30 (DEFAULT_TABLE table=test-table [id=367bdf1b676b45479429e0bcd1abef91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:50.602082 7901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d368db862f44981be7df2464bb2ed30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:50.602314 8032 tablet_service.cc:1467] Processing CreateTablet for tablet 8d368db862f44981be7df2464bb2ed30 (DEFAULT_TABLE table=test-table [id=367bdf1b676b45479429e0bcd1abef91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:50.602615 8032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d368db862f44981be7df2464bb2ed30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:50.604930 8112 tablet_bootstrap.cc:492] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d: Bootstrap starting.
I20260928 14:41:50.605674 8112 tablet_bootstrap.cc:654] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:50.605681 8108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.595989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52772 (local address 127.22.13.65:40441)
0928 14:41:50.596400 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:50.596405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:50.596416 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:41:50.596458 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:50.596460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:50.596477 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:50.596574 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:50.598013 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.598788 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.600173 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.600913 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.600979 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.601061 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:50.605365 (+ 4304us) server_negotiation.cc:299] Negotiation successful
0928 14:41:50.605527 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":115,"threads_started":1}
I20260928 14:41:50.605902 8113 tablet_bootstrap.cc:492] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a: Bootstrap starting.
I20260928 14:41:50.606566 8113 tablet_bootstrap.cc:654] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:50.606472 7770 tablet_service.cc:1467] Processing CreateTablet for tablet 8d368db862f44981be7df2464bb2ed30 (DEFAULT_TABLE table=test-table [id=367bdf1b676b45479429e0bcd1abef91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:50.606863 7770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d368db862f44981be7df2464bb2ed30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:50.606891 8113 log.cc:824] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:50.607678 8113 tablet_bootstrap.cc:492] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a: No bootstrap required, opened a new log
I20260928 14:41:50.607753 8113 ts_tablet_manager.cc:1397] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:41:50.610054 8112 log.cc:824] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:50.610487 8116 tablet_bootstrap.cc:492] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef: Bootstrap starting.
I20260928 14:41:50.610564 8113 raft_consensus.cc:348] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ff0f9c67e634553958fff89455fc83d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38449 } } peers { permanent_uuid: "b8e52dc508824df8a17f648ec109a8ef" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40441 } } peers { permanent_uuid: "91f5cbc8797b4b2bb3823be396b6bc3a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37977 } }
I20260928 14:41:50.610733 8113 raft_consensus.cc:374] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:50.610760 8113 raft_consensus.cc:729] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f5cbc8797b4b2bb3823be396b6bc3a, State: Initialized, Role: FOLLOWER
I20260928 14:41:50.610880 8113 consensus_queue.cc:260] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ff0f9c67e634553958fff89455fc83d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38449 } } peers { permanent_uuid: "b8e52dc508824df8a17f648ec109a8ef" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40441 } } peers { permanent_uuid: "91f5cbc8797b4b2bb3823be396b6bc3a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37977 } }
I20260928 14:41:50.610949 8112 tablet_bootstrap.cc:492] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d: No bootstrap required, opened a new log
I20260928 14:41:50.611013 8112 ts_tablet_manager.cc:1397] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260928 14:41:50.611145 8116 tablet_bootstrap.cc:654] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:50.611145 8113 ts_tablet_manager.cc:1428] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260928 14:41:50.611168 8098 heartbeater.cc:503] Master 127.22.13.126:32775 was elected leader, sending a full tablet report...
I20260928 14:41:50.611466 8116 log.cc:824] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:50.612373 8116 tablet_bootstrap.cc:492] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef: No bootstrap required, opened a new log
I20260928 14:41:50.612430 8116 ts_tablet_manager.cc:1397] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:41:50.613940 8112 raft_consensus.cc:348] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ff0f9c67e634553958fff89455fc83d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38449 } } peers { permanent_uuid: "b8e52dc508824df8a17f648ec109a8ef" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40441 } } peers { permanent_uuid: "91f5cbc8797b4b2bb3823be396b6bc3a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37977 } }
I20260928 14:41:50.614140 8112 raft_consensus.cc:374] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:50.614180 8112 raft_consensus.cc:729] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ff0f9c67e634553958fff89455fc83d, State: Initialized, Role: FOLLOWER
I20260928 14:41:50.614307 8112 consensus_queue.cc:260] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ff0f9c67e634553958fff89455fc83d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38449 } } peers { permanent_uuid: "b8e52dc508824df8a17f648ec109a8ef" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40441 } } peers { permanent_uuid: "91f5cbc8797b4b2bb3823be396b6bc3a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37977 } }
I20260928 14:41:50.614581 8112 ts_tablet_manager.cc:1428] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260928 14:41:50.614625 7967 heartbeater.cc:503] Master 127.22.13.126:32775 was elected leader, sending a full tablet report...
I20260928 14:41:50.615077 8116 raft_consensus.cc:348] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ff0f9c67e634553958fff89455fc83d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38449 } } peers { permanent_uuid: "b8e52dc508824df8a17f648ec109a8ef" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40441 } } peers { permanent_uuid: "91f5cbc8797b4b2bb3823be396b6bc3a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37977 } }
I20260928 14:41:50.615242 8116 raft_consensus.cc:374] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:50.615290 8116 raft_consensus.cc:729] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e52dc508824df8a17f648ec109a8ef, State: Initialized, Role: FOLLOWER
I20260928 14:41:50.615408 8116 consensus_queue.cc:260] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ff0f9c67e634553958fff89455fc83d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38449 } } peers { permanent_uuid: "b8e52dc508824df8a17f648ec109a8ef" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40441 } } peers { permanent_uuid: "91f5cbc8797b4b2bb3823be396b6bc3a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37977 } }
I20260928 14:41:50.615695 8116 ts_tablet_manager.cc:1428] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260928 14:41:50.615742 7836 heartbeater.cc:503] Master 127.22.13.126:32775 was elected leader, sending a full tablet report...
I20260928 14:41:50.663975 8120 raft_consensus.cc:482] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:41:50.664115 8120 raft_consensus.cc:504] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ff0f9c67e634553958fff89455fc83d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38449 } } peers { permanent_uuid: "b8e52dc508824df8a17f648ec109a8ef" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40441 } } peers { permanent_uuid: "91f5cbc8797b4b2bb3823be396b6bc3a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37977 } }
I20260928 14:41:50.664418 8120 leader_election.cc:288] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ff0f9c67e634553958fff89455fc83d (127.22.13.66:38449), 91f5cbc8797b4b2bb3823be396b6bc3a (127.22.13.67:37977)
I20260928 14:41:50.668507 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.664889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:36827 (local address 127.22.13.67:37977)
0928 14:41:50.664961 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:50.664963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:50.664972 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:41:50.665063 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:50.665064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:50.665070 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:50.665107 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:50.665276 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.666016 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.667562 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.668137 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.668217 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.668224 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:50.668393 (+ 169us) server_negotiation.cc:299] Negotiation successful
0928 14:41:50.668434 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 14:41:50.668514 8109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.664710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:40799 (local address 127.22.13.66:38449)
0928 14:41:50.664784 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:50.664786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:50.664834 (+ 48us) server_negotiation.cc:407] Connection header received
0928 14:41:50.664878 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:50.664879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:50.664893 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:50.664972 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:50.665200 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.666011 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.667562 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.668137 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.668217 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.668225 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:50.668394 (+ 169us) server_negotiation.cc:299] Negotiation successful
0928 14:41:50.668434 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 14:41:50.668526 7838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.664641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:38449 (local address 127.22.13.65:40799)
0928 14:41:50.664716 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:50.664729 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:50.664778 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:50.665013 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:50.665014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:50.665016 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:50.665114 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:50.665117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.666105 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.666106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:50.667447 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:50.667451 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.668228 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.668229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:50.668308 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.668311 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:50.668369 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 14:41:50.668423 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":53}
I20260928 14:41:50.668532 8121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.664755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:37977 (local address 127.22.13.65:36827)
0928 14:41:50.664928 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:50.664936 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:50.664995 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:50.665136 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:50.665137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:50.665139 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:50.665199 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:50.665200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.666105 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.666106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:50.667447 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:50.667462 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.668229 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.668229 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:50.668313 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.668316 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:50.668369 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 14:41:50.668421 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":58,"thread_start_us":112,"threads_started":1}
I20260928 14:41:50.669015 8052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d368db862f44981be7df2464bb2ed30" candidate_uuid: "b8e52dc508824df8a17f648ec109a8ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f5cbc8797b4b2bb3823be396b6bc3a" is_pre_election: true
I20260928 14:41:50.669013 7921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d368db862f44981be7df2464bb2ed30" candidate_uuid: "b8e52dc508824df8a17f648ec109a8ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ff0f9c67e634553958fff89455fc83d" is_pre_election: true
I20260928 14:41:50.669190 7921 raft_consensus.cc:2454] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e52dc508824df8a17f648ec109a8ef in term 0.
I20260928 14:41:50.669190 8052 raft_consensus.cc:2454] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8e52dc508824df8a17f648ec109a8ef in term 0.
I20260928 14:41:50.669440 7723 leader_election.cc:302] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f5cbc8797b4b2bb3823be396b6bc3a, b8e52dc508824df8a17f648ec109a8ef; no voters:
I20260928 14:41:50.669586 8120 raft_consensus.cc:2781] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:41:50.669675 8120 raft_consensus.cc:482] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:41:50.669710 8120 raft_consensus.cc:3037] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:50.670477 8120 raft_consensus.cc:504] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ff0f9c67e634553958fff89455fc83d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38449 } } peers { permanent_uuid: "b8e52dc508824df8a17f648ec109a8ef" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40441 } } peers { permanent_uuid: "91f5cbc8797b4b2bb3823be396b6bc3a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37977 } }
I20260928 14:41:50.670621 8120 leader_election.cc:288] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [CANDIDATE]: Term 1 election: Requested vote from peers 1ff0f9c67e634553958fff89455fc83d (127.22.13.66:38449), 91f5cbc8797b4b2bb3823be396b6bc3a (127.22.13.67:37977)
I20260928 14:41:50.670766 7921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d368db862f44981be7df2464bb2ed30" candidate_uuid: "b8e52dc508824df8a17f648ec109a8ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ff0f9c67e634553958fff89455fc83d"
I20260928 14:41:50.670804 8052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d368db862f44981be7df2464bb2ed30" candidate_uuid: "b8e52dc508824df8a17f648ec109a8ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91f5cbc8797b4b2bb3823be396b6bc3a"
I20260928 14:41:50.670854 7921 raft_consensus.cc:3037] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:50.670873 8052 raft_consensus.cc:3037] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:50.671810 8052 raft_consensus.cc:2454] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e52dc508824df8a17f648ec109a8ef in term 1.
I20260928 14:41:50.671810 7921 raft_consensus.cc:2454] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8e52dc508824df8a17f648ec109a8ef in term 1.
I20260928 14:41:50.671993 7723 leader_election.cc:302] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91f5cbc8797b4b2bb3823be396b6bc3a, b8e52dc508824df8a17f648ec109a8ef; no voters:
I20260928 14:41:50.672084 8120 raft_consensus.cc:2781] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:41:50.672243 8120 raft_consensus.cc:686] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [term 1 LEADER]: Becoming Leader. State: Replica: b8e52dc508824df8a17f648ec109a8ef, State: Running, Role: LEADER
I20260928 14:41:50.672346 8120 consensus_queue.cc:237] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ff0f9c67e634553958fff89455fc83d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38449 } } peers { permanent_uuid: "b8e52dc508824df8a17f648ec109a8ef" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40441 } } peers { permanent_uuid: "91f5cbc8797b4b2bb3823be396b6bc3a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37977 } }
I20260928 14:41:50.673502 7647 catalog_manager.cc:5495] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef reported cstate change: term changed from 0 to 1, leader changed from <none> to b8e52dc508824df8a17f648ec109a8ef (127.22.13.65). New cstate: current_term: 1 leader_uuid: "b8e52dc508824df8a17f648ec109a8ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ff0f9c67e634553958fff89455fc83d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8e52dc508824df8a17f648ec109a8ef" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 40441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91f5cbc8797b4b2bb3823be396b6bc3a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37977 } health_report { overall_health: UNKNOWN } } }
W20260928 14:41:50.679417 7968 tablet.cc:2369] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:41:50.697422 8108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.694869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52774 (local address 127.22.13.65:40441)
0928 14:41:50.694927 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:50.694929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:50.694939 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:41:50.695060 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:50.695061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:50.695081 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:50.695154 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:41:50.695303 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.695811 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.696596 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:50.696797 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:50.696874 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:50.697049 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:41:50.697156 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:41:50.697264 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:50.697283 (+ 19us) server_negotiation.cc:299] Negotiation successful
0928 14:41:50.697307 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 14:41:50.699378 8052 raft_consensus.cc:1261] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a [term 1 FOLLOWER]: Refusing update from remote peer b8e52dc508824df8a17f648ec109a8ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:41:50.699429 7921 raft_consensus.cc:1261] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d [term 1 FOLLOWER]: Refusing update from remote peer b8e52dc508824df8a17f648ec109a8ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:41:50.699807 8120 consensus_queue.cc:1035] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "91f5cbc8797b4b2bb3823be396b6bc3a" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 37977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:41:50.699985 8122 consensus_queue.cc:1035] T 8d368db862f44981be7df2464bb2ed30 P b8e52dc508824df8a17f648ec109a8ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ff0f9c67e634553958fff89455fc83d" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:41:50.702792 8127 mvcc.cc:205] Tried to move back new op lower bound from 7334324267821649920 to 7334324267714842624. Current Snapshot: MvccSnapshot[applied={T|T < 7334324267821649920}]
I20260928 14:41:50.703104 8126 mvcc.cc:205] Tried to move back new op lower bound from 7334324267821649920 to 7334324267714842624. Current Snapshot: MvccSnapshot[applied={T|T < 7334324267821649920}]
I20260928 14:41:50.705109 7648 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:41:50.705214 7648 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:41:50.706045 7648 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f254a255beb4e1087fcc38380c35207: Sending DeleteTablet for 3 replicas of tablet 8d368db862f44981be7df2464bb2ed30
I20260928 14:41:50.706394 7901 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d368db862f44981be7df2464bb2ed30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:50 UTC) from {username='slave'} at 127.0.0.1:59490
I20260928 14:41:50.706401 7770 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d368db862f44981be7df2464bb2ed30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:50 UTC) from {username='slave'} at 127.0.0.1:52772
I20260928 14:41:50.706653 8032 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d368db862f44981be7df2464bb2ed30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:50 UTC) from {username='slave'} at 127.0.0.1:48826
I20260928 14:41:50.706921 8136 tablet_replica.cc:330] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a: stopping tablet replica
I20260928 14:41:50.707020 8134 tablet_replica.cc:330] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d: stopping tablet replica
I20260928 14:41:50.707127 8134 raft_consensus.cc:2229] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:41:50.707129 8136 raft_consensus.cc:2229] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:41:50.707269 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 7708
I20260928 14:41:50.707492 8134 raft_consensus.cc:2258] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:50.708070 8136 raft_consensus.cc:2258] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:50.708510 8136 ts_tablet_manager.cc:1905] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:50.708976 8134 ts_tablet_manager.cc:1905] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:50.710400 8136 ts_tablet_manager.cc:1918] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:41:50.710489 8136 log.cc:1196] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/8d368db862f44981be7df2464bb2ed30
I20260928 14:41:50.710768 8136 ts_tablet_manager.cc:1939] T 8d368db862f44981be7df2464bb2ed30 P 91f5cbc8797b4b2bb3823be396b6bc3a: Deleting consensus metadata
I20260928 14:41:50.710863 8134 ts_tablet_manager.cc:1918] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:41:50.710966 8134 log.cc:1196] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/8d368db862f44981be7df2464bb2ed30
I20260928 14:41:50.711233 8134 ts_tablet_manager.cc:1939] T 8d368db862f44981be7df2464bb2ed30 P 1ff0f9c67e634553958fff89455fc83d: Deleting consensus metadata
I20260928 14:41:50.711225 7632 catalog_manager.cc:4841] TS 91f5cbc8797b4b2bb3823be396b6bc3a (127.22.13.67:37977): tablet 8d368db862f44981be7df2464bb2ed30 (table test-table [id=367bdf1b676b45479429e0bcd1abef91]) successfully deleted
I20260928 14:41:50.711728 7632 catalog_manager.cc:4841] TS 1ff0f9c67e634553958fff89455fc83d (127.22.13.66:38449): tablet 8d368db862f44981be7df2464bb2ed30 (table test-table [id=367bdf1b676b45479429e0bcd1abef91]) successfully deleted
W20260928 14:41:50.713305 7635 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:40441 (error 108)
I20260928 14:41:50.713531 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 7839
I20260928 14:41:50.714103 8107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:50.713677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:40441 (local address 127.0.0.1:52790)
0928 14:41:50.714016 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:50.714060 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:40441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314}
W20260928 14:41:50.714334 7635 catalog_manager.cc:4584] TS b8e52dc508824df8a17f648ec109a8ef (127.22.13.65:40441): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d368db862f44981be7df2464bb2ed30: Network error: Client connection negotiation failed: client connection to 127.22.13.65:40441: connect: Connection refused (error 111)
I20260928 14:41:50.718436 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 7970
I20260928 14:41:50.723790 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 7617
2026-09-28T14:41:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (759 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:41:50 dist-test-slave-wwwb krb5kdc[8140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:41:50 dist-test-slave-wwwb krb5kdc[8140](info): set up 2 sockets
Sep 28 14:41:50 dist-test-slave-wwwb krb5kdc[8140](info): commencing operation
krb5kdc: starting...
W20260928 14:41:51.754081 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:41:51 dist-test-slave-wwwb krb5kdc[8140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:41:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:41:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:41:51.870967 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:35705
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:35705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:51.953709 8153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:51.954015 8153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:51.954077 8153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:51.955338 8153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:41:51.955418 8153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:51.955441 8153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:51.955461 8153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:41:51.955487 8153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:41:51.957172 8153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:35705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:35705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:51.957516 8153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:51.957767 8153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:51.962718 8159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:51.962739 8158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:51.962778 8153 server_base.cc:1034] running on GCE node
W20260928 14:41:51.962724 8161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:51.963179 8153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:51.963459 8153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:51.964605 8153 hybrid_clock.cc:648] HybridClock initialized: now 1790606511964592 us; error 30 us; skew 500 ppm
Sep 28 14:41:51 dist-test-slave-wwwb krb5kdc[8140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606511, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:51.970445 8153 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:41:51.970898 8153 webserver.cc:460] Webserver started at http://127.22.13.126:36259/ using document root <none> and password file <none>
I20260928 14:41:51.971122 8153 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:51.971164 8153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:51.971263 8153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:51.972354 8153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "17b89f9d3d5a40ff864f83acf96e8757"
format_stamp: "Formatted at 2026-09-28 14:41:51 on dist-test-slave-wwwb"
server_key: "f63c5623953f248b025a9d11c19ca06a"
server_key_iv: "803cc2e249714bbeecd1836a582c09ff"
server_key_version: "encryptionkey@0"
I20260928 14:41:51.972668 8153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "17b89f9d3d5a40ff864f83acf96e8757"
format_stamp: "Formatted at 2026-09-28 14:41:51 on dist-test-slave-wwwb"
server_key: "f63c5623953f248b025a9d11c19ca06a"
server_key_iv: "803cc2e249714bbeecd1836a582c09ff"
server_key_version: "encryptionkey@0"
I20260928 14:41:51.974063 8153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 14:41:51.974833 8168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:51.975009 8153 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:51.975118 8153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "17b89f9d3d5a40ff864f83acf96e8757"
format_stamp: "Formatted at 2026-09-28 14:41:51 on dist-test-slave-wwwb"
server_key: "f63c5623953f248b025a9d11c19ca06a"
server_key_iv: "803cc2e249714bbeecd1836a582c09ff"
server_key_version: "encryptionkey@0"
I20260928 14:41:51.975209 8153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:51.994207 8153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:51.996169 8153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:51.996361 8153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:52.000587 8153 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:35705
I20260928 14:41:52.000626 8220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:35705 every 8 connection(s)
I20260928 14:41:52.001003 8153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:41:52.002068 8221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:52.004608 8221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757: Bootstrap starting.
I20260928 14:41:52.005259 8221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:52.005604 8221 log.cc:824] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:52.006441 8221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757: No bootstrap required, opened a new log
I20260928 14:41:52.006614 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 8153
I20260928 14:41:52.006740 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:41:52.006950 22581 external_mini_cluster.cc:1428] Setting key dc167c09bf150ea12870b73bebb68a40
I20260928 14:41:52.009378 8221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b89f9d3d5a40ff864f83acf96e8757" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35705 } }
I20260928 14:41:52.009593 8221 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:52.009634 8221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17b89f9d3d5a40ff864f83acf96e8757, State: Initialized, Role: FOLLOWER
I20260928 14:41:52.009779 8221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b89f9d3d5a40ff864f83acf96e8757" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35705 } }
I20260928 14:41:52.009867 8221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:41:52.009891 8221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:41:52.009985 8221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:52.010713 8221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b89f9d3d5a40ff864f83acf96e8757" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35705 } }
I20260928 14:41:52.010824 8221 leader_election.cc:302] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17b89f9d3d5a40ff864f83acf96e8757; no voters:
I20260928 14:41:52.010993 8221 leader_election.cc:288] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:41:52.011070 8226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:41:52.011228 8226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [term 1 LEADER]: Becoming Leader. State: Replica: 17b89f9d3d5a40ff864f83acf96e8757, State: Running, Role: LEADER
I20260928 14:41:52.011260 8221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:41:52.011322 8226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b89f9d3d5a40ff864f83acf96e8757" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35705 } }
I20260928 14:41:52.011792 8227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17b89f9d3d5a40ff864f83acf96e8757" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b89f9d3d5a40ff864f83acf96e8757" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35705 } } }
I20260928 14:41:52.011873 8227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:52.012338 8234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:41:52.012096 8228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17b89f9d3d5a40ff864f83acf96e8757. Latest consensus state: current_term: 1 leader_uuid: "17b89f9d3d5a40ff864f83acf96e8757" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17b89f9d3d5a40ff864f83acf96e8757" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35705 } } }
I20260928 14:41:52.012501 8228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:52.012924 8234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:41:52.014758 8234 catalog_manager.cc:1347] Generated new cluster ID: 2510a11d0b8b4eab883a1ad715466df8
I20260928 14:41:52.014819 8234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:41:52 dist-test-slave-wwwb krb5kdc[8140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:41:52.021569 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.007562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35046 (local address 127.22.13.126:35705)
0928 14:41:52.007767 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:52.007776 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:52.007815 (+ 39us) server_negotiation.cc:407] Connection header received
0928 14:41:52.009320 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:52.009331 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:52.009408 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:52.009581 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:52.010457 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.012106 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.013263 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.013911 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.014058 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.017058 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:52.017063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:52.017063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:52.017077 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:52.019239 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:52.019683 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:52.019686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:52.019686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:52.019809 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:52.020027 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:52.020028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:52.020028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:52.020426 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:52.021100 (+ 674us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:52.021185 (+ 85us) server_negotiation.cc:299] Negotiation successful
0928 14:41:52.021350 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260928 14:41:52.038425 8234 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:41:52.039237 8234 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:41:52.048110 8234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757: Generated new TSK 0
I20260928 14:41:52.048367 8234 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:41:52.077203 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35705
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:52.161392 8247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:52.161656 8247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:52.161700 8247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:52.162988 8247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:52.163070 8247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:52.163117 8247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:41:52.164637 8247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:52.164929 8247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:52.165203 8247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:52.170341 8255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:52.170411 8253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:52.170316 8252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:52.170533 8247 server_base.cc:1034] running on GCE node
I20260928 14:41:52.170786 8247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:52.171078 8247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:52.172254 8247 hybrid_clock.cc:648] HybridClock initialized: now 1790606512172232 us; error 39 us; skew 500 ppm
Sep 28 14:41:52 dist-test-slave-wwwb krb5kdc[8140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606512, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:52.174963 8247 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:41:52.175434 8247 webserver.cc:460] Webserver started at http://127.22.13.65:33885/ using document root <none> and password file <none>
I20260928 14:41:52.175688 8247 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:52.175736 8247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:52.175915 8247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:52.177067 8247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "6d7af6b22da4469e81c05891b379afdc"
format_stamp: "Formatted at 2026-09-28 14:41:52 on dist-test-slave-wwwb"
server_key: "8eaacf202d9dfd9552684783a2831746"
server_key_iv: "d523163e03f2ce68cba60251f9fd9b29"
server_key_version: "encryptionkey@0"
I20260928 14:41:52.177438 8247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "6d7af6b22da4469e81c05891b379afdc"
format_stamp: "Formatted at 2026-09-28 14:41:52 on dist-test-slave-wwwb"
server_key: "8eaacf202d9dfd9552684783a2831746"
server_key_iv: "d523163e03f2ce68cba60251f9fd9b29"
server_key_version: "encryptionkey@0"
I20260928 14:41:52.178966 8247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:41:52.179795 8262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:52.179983 8247 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:52.180080 8247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "6d7af6b22da4469e81c05891b379afdc"
format_stamp: "Formatted at 2026-09-28 14:41:52 on dist-test-slave-wwwb"
server_key: "8eaacf202d9dfd9552684783a2831746"
server_key_iv: "d523163e03f2ce68cba60251f9fd9b29"
server_key_version: "encryptionkey@0"
I20260928 14:41:52.180159 8247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:52.196269 8247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:52.198469 8247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:52.198645 8247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:52.198931 8247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:52.199290 8247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:52.199348 8247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:52.199383 8247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:52.199412 8247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:52.205478 8247 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:37651
I20260928 14:41:52.205523 8375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:37651 every 8 connection(s)
I20260928 14:41:52.205889 8247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
Sep 28 14:41:52 dist-test-slave-wwwb krb5kdc[8140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606512, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:41:52.213243 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 8247
I20260928 14:41:52.213387 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:41:52.213647 22581 external_mini_cluster.cc:1428] Setting key a480e50a07b7d7bf78426da988a93d6c
I20260928 14:41:52.215986 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.206740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:52713 (local address 127.22.13.126:35705)
0928 14:41:52.206815 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:52.206817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:52.206991 (+ 174us) server_negotiation.cc:407] Connection header received
0928 14:41:52.207489 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:52.207490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:52.207499 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:52.207564 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:52.208544 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.209058 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.210151 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.210366 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.210422 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.212793 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:52.212795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:52.212796 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:52.212803 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:52.214497 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:52.214903 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:52.214904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:52.214905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:52.214971 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:52.215223 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:52.215224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:52.215225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:52.215382 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:52.215472 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:52.215737 (+ 265us) server_negotiation.cc:299] Negotiation successful
0928 14:41:52.215847 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 14:41:52.216188 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.206659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:35705 (local address 127.22.13.65:52713)
0928 14:41:52.206854 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:52.206897 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:52.207402 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:52.207629 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:52.207632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:52.208231 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:52.208447 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.208453 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.209159 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.209159 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.210072 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.210075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.210445 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.210445 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.210586 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.211092 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:41:52.211101 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:52.212690 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:52.214582 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:52.214590 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:52.214592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:52.214825 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:52.215040 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:52.215041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:52.215041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:52.215162 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:52.215469 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:52.215474 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:52.215607 (+ 133us) client_negotiation.cc:769] Sending connection context
0928 14:41:52.215694 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 14:41:52.215907 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
I20260928 14:41:52.216806 8376 heartbeater.cc:347] Connected to a master server at 127.22.13.126:35705
I20260928 14:41:52.216915 8376 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:52.217120 8376 heartbeater.cc:511] Master 127.22.13.126:35705 requested a full tablet report, sending...
I20260928 14:41:52.217682 8185 ts_manager.cc:194] Registered new tserver with Master: 6d7af6b22da4469e81c05891b379afdc (127.22.13.65:37651)
I20260928 14:41:52.218389 8185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:52713
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:41:52.238811 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35705
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:52.327618 8381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:52.327837 8381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:52.327883 8381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:52.329034 8381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:52.329072 8381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:52.329103 8381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:41:52.330675 8381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:52.330888 8381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:52.331085 8381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:52.336355 8386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:52.336437 8387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:52.336354 8389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:52.336638 8381 server_base.cc:1034] running on GCE node
I20260928 14:41:52.336838 8381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:52.337100 8381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:52.338243 8381 hybrid_clock.cc:648] HybridClock initialized: now 1790606512338230 us; error 35 us; skew 500 ppm
Sep 28 14:41:52 dist-test-slave-wwwb krb5kdc[8140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606512, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:52.340983 8381 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:41:52.341390 8381 webserver.cc:460] Webserver started at http://127.22.13.66:46275/ using document root <none> and password file <none>
I20260928 14:41:52.341598 8381 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:52.341640 8381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:52.341734 8381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:52.342893 8381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "2bac1f51acec42c9b203a91a60d1d678"
format_stamp: "Formatted at 2026-09-28 14:41:52 on dist-test-slave-wwwb"
server_key: "394adab8141fc893f9644aa455aae5b1"
server_key_iv: "b6284368416b29251e8b655d22f6da81"
server_key_version: "encryptionkey@0"
I20260928 14:41:52.343209 8381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "2bac1f51acec42c9b203a91a60d1d678"
format_stamp: "Formatted at 2026-09-28 14:41:52 on dist-test-slave-wwwb"
server_key: "394adab8141fc893f9644aa455aae5b1"
server_key_iv: "b6284368416b29251e8b655d22f6da81"
server_key_version: "encryptionkey@0"
I20260928 14:41:52.344571 8381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:41:52.345266 8396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:52.345458 8381 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:41:52.345531 8381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "2bac1f51acec42c9b203a91a60d1d678"
format_stamp: "Formatted at 2026-09-28 14:41:52 on dist-test-slave-wwwb"
server_key: "394adab8141fc893f9644aa455aae5b1"
server_key_iv: "b6284368416b29251e8b655d22f6da81"
server_key_version: "encryptionkey@0"
I20260928 14:41:52.345598 8381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:52.362147 8381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:52.364521 8381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:52.364640 8381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:52.364822 8381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:52.365151 8381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:52.365186 8381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:52.365208 8381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:52.365254 8381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:52.371311 8381 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:36647
I20260928 14:41:52.371356 8509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:36647 every 8 connection(s)
I20260928 14:41:52.371748 8381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:41:52.374243 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 8381
I20260928 14:41:52.374352 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:41:52.374607 22581 external_mini_cluster.cc:1428] Setting key 1360f0923e35e2b9d34e608e7f80cf9b
Sep 28 14:41:52 dist-test-slave-wwwb krb5kdc[8140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606512, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
I20260928 14:41:52.382797 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.372530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:58099 (local address 127.22.13.126:35705)
0928 14:41:52.372600 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:52.372603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:52.372863 (+ 260us) server_negotiation.cc:407] Connection header received
0928 14:41:52.373359 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:52.373360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:52.373370 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:52.373425 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:52.374641 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.375387 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.376518 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.376801 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.376862 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.379808 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:52.379810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:52.379810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:52.379819 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:52.381253 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:52.381716 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:52.381718 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:52.381718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:52.381780 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:52.382109 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:52.382110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:52.382110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:52.382237 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:52.382315 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:52.382617 (+ 302us) server_negotiation.cc:299] Negotiation successful
0928 14:41:52.382701 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 14:41:52.383009 8512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.372524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:35705 (local address 127.22.13.66:58099)
0928 14:41:52.372733 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:52.372774 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:52.373263 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:52.373509 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:52.373512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:52.374226 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:52.374532 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.374547 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.375478 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.375479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.376385 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.376388 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.376894 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.376896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.377065 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.377729 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:41:52.377738 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:52.379678 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:52.381359 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:52.381368 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:52.381370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:52.381615 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:52.381849 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:52.381850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:52.381851 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:52.382033 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:52.382306 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:52.382312 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:52.382474 (+ 162us) client_negotiation.cc:769] Sending connection context
0928 14:41:52.382581 (+ 107us) client_negotiation.cc:240] Negotiation successful
0928 14:41:52.382758 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":117,"threads_started":1}
I20260928 14:41:52.383546 8510 heartbeater.cc:347] Connected to a master server at 127.22.13.126:35705
I20260928 14:41:52.383666 8510 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:52.383888 8510 heartbeater.cc:511] Master 127.22.13.126:35705 requested a full tablet report, sending...
I20260928 14:41:52.384326 8185 ts_manager.cc:194] Registered new tserver with Master: 2bac1f51acec42c9b203a91a60d1d678 (127.22.13.66:36647)
I20260928 14:41:52.384713 8185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:58099
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:41:52.398864 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35705
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:52.482218 8515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:52.482437 8515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:52.482463 8515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:52.483824 8515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:52.483876 8515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:52.483927 8515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:41:52.485494 8515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:52.485783 8515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:52.486109 8515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:52.491298 8520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:52.491292 8523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:52.491511 8515 server_base.cc:1034] running on GCE node
W20260928 14:41:52.491292 8521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:52.491794 8515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:52.492071 8515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:52.493247 8515 hybrid_clock.cc:648] HybridClock initialized: now 1790606512493231 us; error 28 us; skew 500 ppm
Sep 28 14:41:52 dist-test-slave-wwwb krb5kdc[8140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606512, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:52.495814 8515 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:41:52.496317 8515 webserver.cc:460] Webserver started at http://127.22.13.67:43169/ using document root <none> and password file <none>
I20260928 14:41:52.496559 8515 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:52.496619 8515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:52.496785 8515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:52.497885 8515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "e2896a5abff440f2b53412b83938c1da"
format_stamp: "Formatted at 2026-09-28 14:41:52 on dist-test-slave-wwwb"
server_key: "4dcbd2d7e4beda24ff8af86b8e02a941"
server_key_iv: "6b8acd94fec40bd6b19875a3e72fa741"
server_key_version: "encryptionkey@0"
I20260928 14:41:52.498322 8515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "e2896a5abff440f2b53412b83938c1da"
format_stamp: "Formatted at 2026-09-28 14:41:52 on dist-test-slave-wwwb"
server_key: "4dcbd2d7e4beda24ff8af86b8e02a941"
server_key_iv: "6b8acd94fec40bd6b19875a3e72fa741"
server_key_version: "encryptionkey@0"
I20260928 14:41:52.499758 8515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:41:52.500491 8530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:52.500670 8515 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:52.500779 8515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "e2896a5abff440f2b53412b83938c1da"
format_stamp: "Formatted at 2026-09-28 14:41:52 on dist-test-slave-wwwb"
server_key: "4dcbd2d7e4beda24ff8af86b8e02a941"
server_key_iv: "6b8acd94fec40bd6b19875a3e72fa741"
server_key_version: "encryptionkey@0"
I20260928 14:41:52.500856 8515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:52.511583 8515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:52.514060 8515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:52.514237 8515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:52.514478 8515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:52.514806 8515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:52.514863 8515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:52.514906 8515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:52.514932 8515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:52.520895 8515 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:43103
I20260928 14:41:52.521013 8643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:43103 every 8 connection(s)
I20260928 14:41:52.521289 8515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:41:52.523824 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 8515
I20260928 14:41:52.523908 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:41:52.524111 22581 external_mini_cluster.cc:1428] Setting key 67e1f8fdce94f00ed5a0d241a428836b
Sep 28 14:41:52 dist-test-slave-wwwb krb5kdc[8140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606512, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:41:52.530956 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.522094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:42329 (local address 127.22.13.126:35705)
0928 14:41:52.522155 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:52.522157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:52.522405 (+ 248us) server_negotiation.cc:407] Connection header received
0928 14:41:52.522953 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:52.522954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:52.522966 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:52.523035 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:52.524050 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.524840 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.526014 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.526235 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.526323 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.528442 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:52.528443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:52.528443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:52.528450 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:52.529661 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:52.530131 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:52.530131 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:52.530132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:52.530186 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:52.530411 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:52.530411 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:52.530412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:52.530519 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:52.530570 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:52.530816 (+ 246us) server_negotiation.cc:299] Negotiation successful
0928 14:41:52.530888 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 14:41:52.531147 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.522107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:35705 (local address 127.22.13.67:42329)
0928 14:41:52.522285 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:52.522322 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:52.522839 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:52.523097 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:52.523101 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:52.523692 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:52.523937 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.523944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.524930 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.524931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.525883 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.525885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.526336 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.526337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.526477 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.526956 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:41:52.526965 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:52.528344 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:52.529742 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:52.529748 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:52.529750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:52.530060 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:52.530249 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:52.530250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:52.530250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:52.530353 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:52.530573 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:52.530577 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:52.530701 (+ 124us) client_negotiation.cc:769] Sending connection context
0928 14:41:52.530777 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 14:41:52.530899 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":87,"threads_started":1}
I20260928 14:41:52.531659 8644 heartbeater.cc:347] Connected to a master server at 127.22.13.126:35705
I20260928 14:41:52.531739 8644 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:52.531944 8644 heartbeater.cc:511] Master 127.22.13.126:35705 requested a full tablet report, sending...
I20260928 14:41:52.532325 8185 ts_manager.cc:194] Registered new tserver with Master: e2896a5abff440f2b53412b83938c1da (127.22.13.67:43103)
I20260928 14:41:52.532718 8185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:42329
I20260928 14:41:52.535931 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:41:52.543955 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.537642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35058 (local address 127.22.13.126:35705)
0928 14:41:52.537702 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:52.537704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:52.537794 (+ 90us) server_negotiation.cc:407] Connection header received
0928 14:41:52.537848 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:52.537848 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:52.537858 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:52.537923 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:52.538546 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.539019 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.539722 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.539904 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.539954 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.541321 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:52.541321 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:52.541322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:52.541326 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:52.542631 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:52.543070 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:52.543072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:52.543072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:52.543136 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:52.543372 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:52.543372 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:52.543373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:52.543518 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:52.543639 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:52.543782 (+ 143us) server_negotiation.cc:299] Negotiation successful
0928 14:41:52.543848 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:41:52.545806 8185 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:41:52.547605 8185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:41:52.554692 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.550668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43356 (local address 127.22.13.66:36647)
0928 14:41:52.550873 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:52.550877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:52.550887 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:41:52.550926 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:52.550928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:52.550944 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:52.551240 (+ 296us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:52.551322 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.552306 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.553444 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.554235 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.554324 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.554401 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:52.554553 (+ 152us) server_negotiation.cc:299] Negotiation successful
0928 14:41:52.554603 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":87,"threads_started":1}
I20260928 14:41:52.554698 8657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.550371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:36647 (local address 127.0.0.1:43356)
0928 14:41:52.550455 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:52.550466 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:52.550529 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:52.551120 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:52.551121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:52.551123 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:52.551169 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.551171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.552414 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.552415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.553355 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.553357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.554332 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.554333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.554424 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.554440 (+ 16us) client_negotiation.cc:769] Sending connection context
0928 14:41:52.554524 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 14:41:52.554610 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"thread_start_us":52,"threads_started":1}
I20260928 14:41:52.554881 8658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.550507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:37651 (local address 127.0.0.1:48952)
0928 14:41:52.550664 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:52.550674 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:52.550728 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:52.551076 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:52.551077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:52.551079 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:52.551138 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.551140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.552027 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.552028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.553450 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.553452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.554438 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.554439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.554574 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.554579 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:41:52.554632 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 14:41:52.554680 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":61,"thread_start_us":37,"threads_started":1}
I20260928 14:41:52.555086 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.550630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48952 (local address 127.22.13.65:37651)
0928 14:41:52.550818 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:52.550822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:52.550832 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:41:52.550871 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:52.550873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:52.550889 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:52.550962 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:52.551217 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.551908 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.553540 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.554280 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.554340 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.554401 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:52.554740 (+ 339us) server_negotiation.cc:299] Negotiation successful
0928 14:41:52.555006 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":82,"threads_started":1}
I20260928 14:41:52.555447 8655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.550132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:43103 (local address 127.0.0.1:47608)
0928 14:41:52.550258 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:52.550301 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:52.550376 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:52.550662 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:52.550665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:52.550667 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:52.550761 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.550763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.552141 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.552142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.553449 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.553452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.555230 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.555231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.555297 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.555300 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:52.555341 (+ 41us) client_negotiation.cc:240] Negotiation successful
0928 14:41:52.555385 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":57,"thread_start_us":78,"threads_started":1}
I20260928 14:41:52.555534 8444 tablet_service.cc:1467] Processing CreateTablet for tablet a6a90edc6662423e89d979320ccfb2dd (DEFAULT_TABLE table=test-table [id=e076b7f00be44c53b5e801262965b5b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:52.555794 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.550224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47608 (local address 127.22.13.67:43103)
0928 14:41:52.550413 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:52.550418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:52.550431 (+ 13us) server_negotiation.cc:407] Connection header received
0928 14:41:52.550476 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:52.550480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:52.550497 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:52.550576 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:52.550890 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.552030 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.554019 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.554682 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.554751 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.555641 (+ 890us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:52.555668 (+ 27us) server_negotiation.cc:299] Negotiation successful
0928 14:41:52.555715 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":125,"threads_started":1}
I20260928 14:41:52.555955 8444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6a90edc6662423e89d979320ccfb2dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:52.556387 8578 tablet_service.cc:1467] Processing CreateTablet for tablet a6a90edc6662423e89d979320ccfb2dd (DEFAULT_TABLE table=test-table [id=e076b7f00be44c53b5e801262965b5b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:52.556697 8578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6a90edc6662423e89d979320ccfb2dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:52.558619 8310 tablet_service.cc:1467] Processing CreateTablet for tablet a6a90edc6662423e89d979320ccfb2dd (DEFAULT_TABLE table=test-table [id=e076b7f00be44c53b5e801262965b5b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:52.558959 8310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6a90edc6662423e89d979320ccfb2dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:52.559106 8661 tablet_bootstrap.cc:492] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678: Bootstrap starting.
I20260928 14:41:52.559697 8662 tablet_bootstrap.cc:492] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da: Bootstrap starting.
I20260928 14:41:52.560017 8661 tablet_bootstrap.cc:654] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:52.560395 8662 tablet_bootstrap.cc:654] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:52.560601 8661 log.cc:824] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:52.561007 8662 log.cc:824] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:52.561474 8661 tablet_bootstrap.cc:492] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678: No bootstrap required, opened a new log
I20260928 14:41:52.561545 8661 ts_tablet_manager.cc:1397] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:41:52.561825 8662 tablet_bootstrap.cc:492] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da: No bootstrap required, opened a new log
I20260928 14:41:52.561900 8662 ts_tablet_manager.cc:1397] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:41:52.562522 8665 tablet_bootstrap.cc:492] T a6a90edc6662423e89d979320ccfb2dd P 6d7af6b22da4469e81c05891b379afdc: Bootstrap starting.
I20260928 14:41:52.563274 8665 tablet_bootstrap.cc:654] T a6a90edc6662423e89d979320ccfb2dd P 6d7af6b22da4469e81c05891b379afdc: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:52.563587 8662 raft_consensus.cc:348] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2896a5abff440f2b53412b83938c1da" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 43103 } } peers { permanent_uuid: "2bac1f51acec42c9b203a91a60d1d678" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36647 } } peers { permanent_uuid: "6d7af6b22da4469e81c05891b379afdc" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37651 } }
I20260928 14:41:52.563753 8662 raft_consensus.cc:374] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:52.563776 8662 raft_consensus.cc:729] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2896a5abff440f2b53412b83938c1da, State: Initialized, Role: FOLLOWER
I20260928 14:41:52.563913 8662 consensus_queue.cc:260] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2896a5abff440f2b53412b83938c1da" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 43103 } } peers { permanent_uuid: "2bac1f51acec42c9b203a91a60d1d678" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36647 } } peers { permanent_uuid: "6d7af6b22da4469e81c05891b379afdc" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37651 } }
I20260928 14:41:52.563971 8661 raft_consensus.cc:348] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2896a5abff440f2b53412b83938c1da" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 43103 } } peers { permanent_uuid: "2bac1f51acec42c9b203a91a60d1d678" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36647 } } peers { permanent_uuid: "6d7af6b22da4469e81c05891b379afdc" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37651 } }
I20260928 14:41:52.564157 8661 raft_consensus.cc:374] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:52.564205 8661 raft_consensus.cc:729] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bac1f51acec42c9b203a91a60d1d678, State: Initialized, Role: FOLLOWER
I20260928 14:41:52.564200 8644 heartbeater.cc:503] Master 127.22.13.126:35705 was elected leader, sending a full tablet report...
I20260928 14:41:52.564201 8662 ts_tablet_manager.cc:1428] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:41:52.564365 8665 log.cc:824] T a6a90edc6662423e89d979320ccfb2dd P 6d7af6b22da4469e81c05891b379afdc: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:52.564364 8661 consensus_queue.cc:260] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2896a5abff440f2b53412b83938c1da" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 43103 } } peers { permanent_uuid: "2bac1f51acec42c9b203a91a60d1d678" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36647 } } peers { permanent_uuid: "6d7af6b22da4469e81c05891b379afdc" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37651 } }
I20260928 14:41:52.564671 8661 ts_tablet_manager.cc:1428] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:41:52.564677 8510 heartbeater.cc:503] Master 127.22.13.126:35705 was elected leader, sending a full tablet report...
I20260928 14:41:52.565582 8665 tablet_bootstrap.cc:492] T a6a90edc6662423e89d979320ccfb2dd P 6d7af6b22da4469e81c05891b379afdc: No bootstrap required, opened a new log
I20260928 14:41:52.565696 8665 ts_tablet_manager.cc:1397] T a6a90edc6662423e89d979320ccfb2dd P 6d7af6b22da4469e81c05891b379afdc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:41:52.567329 8665 raft_consensus.cc:348] T a6a90edc6662423e89d979320ccfb2dd P 6d7af6b22da4469e81c05891b379afdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2896a5abff440f2b53412b83938c1da" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 43103 } } peers { permanent_uuid: "2bac1f51acec42c9b203a91a60d1d678" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36647 } } peers { permanent_uuid: "6d7af6b22da4469e81c05891b379afdc" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37651 } }
I20260928 14:41:52.567478 8665 raft_consensus.cc:374] T a6a90edc6662423e89d979320ccfb2dd P 6d7af6b22da4469e81c05891b379afdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:52.567502 8665 raft_consensus.cc:729] T a6a90edc6662423e89d979320ccfb2dd P 6d7af6b22da4469e81c05891b379afdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7af6b22da4469e81c05891b379afdc, State: Initialized, Role: FOLLOWER
I20260928 14:41:52.567654 8665 consensus_queue.cc:260] T a6a90edc6662423e89d979320ccfb2dd P 6d7af6b22da4469e81c05891b379afdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2896a5abff440f2b53412b83938c1da" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 43103 } } peers { permanent_uuid: "2bac1f51acec42c9b203a91a60d1d678" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36647 } } peers { permanent_uuid: "6d7af6b22da4469e81c05891b379afdc" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37651 } }
I20260928 14:41:52.567972 8665 ts_tablet_manager.cc:1428] T a6a90edc6662423e89d979320ccfb2dd P 6d7af6b22da4469e81c05891b379afdc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:41:52.567979 8376 heartbeater.cc:503] Master 127.22.13.126:35705 was elected leader, sending a full tablet report...
I20260928 14:41:52.592276 8666 raft_consensus.cc:482] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:41:52.592451 8666 raft_consensus.cc:504] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2896a5abff440f2b53412b83938c1da" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 43103 } } peers { permanent_uuid: "2bac1f51acec42c9b203a91a60d1d678" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36647 } } peers { permanent_uuid: "6d7af6b22da4469e81c05891b379afdc" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37651 } }
I20260928 14:41:52.592828 8666 leader_election.cc:288] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2bac1f51acec42c9b203a91a60d1d678 (127.22.13.66:36647), 6d7af6b22da4469e81c05891b379afdc (127.22.13.65:37651)
I20260928 14:41:52.596019 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.593128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:53687 (local address 127.22.13.66:36647)
0928 14:41:52.593178 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:52.593180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:52.593193 (+ 13us) server_negotiation.cc:407] Connection header received
0928 14:41:52.593280 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:52.593281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:52.593294 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:52.593392 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:52.593612 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.594231 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.595254 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.595646 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.595705 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.595712 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:52.595897 (+ 185us) server_negotiation.cc:299] Negotiation successful
0928 14:41:52.595940 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 14:41:52.596071 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.593265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:47517 (local address 127.22.13.65:37651)
0928 14:41:52.593327 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:52.593330 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:52.593470 (+ 140us) server_negotiation.cc:407] Connection header received
0928 14:41:52.593520 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:52.593522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:52.593535 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:52.593588 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:52.593752 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.594228 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.595256 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.595658 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.595753 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.595761 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:52.595935 (+ 174us) server_negotiation.cc:299] Negotiation successful
0928 14:41:52.595997 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260928 14:41:52.596052 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.593055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:36647 (local address 127.22.13.67:53687)
0928 14:41:52.593120 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:52.593132 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:52.593186 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:52.593398 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:52.593399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:52.593400 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:52.593505 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.593507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.594330 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.594331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.595152 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.595154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.595737 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.595738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.595827 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.595830 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:52.595872 (+ 42us) client_negotiation.cc:240] Negotiation successful
0928 14:41:52.595932 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":33}
I20260928 14:41:52.596065 8670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.593184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:37651 (local address 127.22.13.67:47517)
0928 14:41:52.593384 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:52.593395 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:52.593463 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:52.593608 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:52.593609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:52.593610 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:52.593673 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.593675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.594335 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.594336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.595153 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.595155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.595780 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.595781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.595834 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.595837 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:52.595918 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 14:41:52.595957 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":2,"thread_start_us":144,"threads_started":1}
I20260928 14:41:52.596621 8464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6a90edc6662423e89d979320ccfb2dd" candidate_uuid: "e2896a5abff440f2b53412b83938c1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bac1f51acec42c9b203a91a60d1d678" is_pre_election: true
I20260928 14:41:52.596596 8330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6a90edc6662423e89d979320ccfb2dd" candidate_uuid: "e2896a5abff440f2b53412b83938c1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d7af6b22da4469e81c05891b379afdc" is_pre_election: true
I20260928 14:41:52.596772 8330 raft_consensus.cc:2454] T a6a90edc6662423e89d979320ccfb2dd P 6d7af6b22da4469e81c05891b379afdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2896a5abff440f2b53412b83938c1da in term 0.
I20260928 14:41:52.596772 8464 raft_consensus.cc:2454] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2896a5abff440f2b53412b83938c1da in term 0.
I20260928 14:41:52.596992 8534 leader_election.cc:302] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d7af6b22da4469e81c05891b379afdc, e2896a5abff440f2b53412b83938c1da; no voters:
I20260928 14:41:52.597123 8666 raft_consensus.cc:2781] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:41:52.597218 8666 raft_consensus.cc:482] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:41:52.597251 8666 raft_consensus.cc:3037] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:52.598057 8666 raft_consensus.cc:504] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2896a5abff440f2b53412b83938c1da" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 43103 } } peers { permanent_uuid: "2bac1f51acec42c9b203a91a60d1d678" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36647 } } peers { permanent_uuid: "6d7af6b22da4469e81c05891b379afdc" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37651 } }
I20260928 14:41:52.598199 8666 leader_election.cc:288] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [CANDIDATE]: Term 1 election: Requested vote from peers 2bac1f51acec42c9b203a91a60d1d678 (127.22.13.66:36647), 6d7af6b22da4469e81c05891b379afdc (127.22.13.65:37651)
I20260928 14:41:52.598559 8330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6a90edc6662423e89d979320ccfb2dd" candidate_uuid: "e2896a5abff440f2b53412b83938c1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d7af6b22da4469e81c05891b379afdc"
I20260928 14:41:52.598556 8464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6a90edc6662423e89d979320ccfb2dd" candidate_uuid: "e2896a5abff440f2b53412b83938c1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bac1f51acec42c9b203a91a60d1d678"
I20260928 14:41:52.598644 8464 raft_consensus.cc:3037] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:52.598644 8330 raft_consensus.cc:3037] T a6a90edc6662423e89d979320ccfb2dd P 6d7af6b22da4469e81c05891b379afdc [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:52.599611 8464 raft_consensus.cc:2454] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2896a5abff440f2b53412b83938c1da in term 1.
I20260928 14:41:52.599611 8330 raft_consensus.cc:2454] T a6a90edc6662423e89d979320ccfb2dd P 6d7af6b22da4469e81c05891b379afdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2896a5abff440f2b53412b83938c1da in term 1.
I20260928 14:41:52.599783 8534 leader_election.cc:302] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d7af6b22da4469e81c05891b379afdc, e2896a5abff440f2b53412b83938c1da; no voters:
I20260928 14:41:52.599888 8666 raft_consensus.cc:2781] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:41:52.600064 8666 raft_consensus.cc:686] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [term 1 LEADER]: Becoming Leader. State: Replica: e2896a5abff440f2b53412b83938c1da, State: Running, Role: LEADER
I20260928 14:41:52.600184 8666 consensus_queue.cc:237] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2896a5abff440f2b53412b83938c1da" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 43103 } } peers { permanent_uuid: "2bac1f51acec42c9b203a91a60d1d678" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36647 } } peers { permanent_uuid: "6d7af6b22da4469e81c05891b379afdc" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37651 } }
I20260928 14:41:52.601086 8185 catalog_manager.cc:5495] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da reported cstate change: term changed from 0 to 1, leader changed from <none> to e2896a5abff440f2b53412b83938c1da (127.22.13.67). New cstate: current_term: 1 leader_uuid: "e2896a5abff440f2b53412b83938c1da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2896a5abff440f2b53412b83938c1da" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 43103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2bac1f51acec42c9b203a91a60d1d678" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d7af6b22da4469e81c05891b379afdc" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37651 } health_report { overall_health: UNKNOWN } } }
I20260928 14:41:52.613333 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.610804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47610 (local address 127.22.13.67:43103)
0928 14:41:52.610888 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:52.610890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:52.610899 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:41:52.610935 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:52.610936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:52.610953 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:52.611027 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:41:52.611228 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.611702 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.612564 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.612751 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.612806 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.612981 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:41:52.613076 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:41:52.613154 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:52.613243 (+ 89us) server_negotiation.cc:299] Negotiation successful
0928 14:41:52.613254 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 14:41:52.615342 8330 raft_consensus.cc:1261] T a6a90edc6662423e89d979320ccfb2dd P 6d7af6b22da4469e81c05891b379afdc [term 1 FOLLOWER]: Refusing update from remote peer e2896a5abff440f2b53412b83938c1da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:41:52.615351 8464 raft_consensus.cc:1261] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678 [term 1 FOLLOWER]: Refusing update from remote peer e2896a5abff440f2b53412b83938c1da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:41:52.615727 8666 consensus_queue.cc:1035] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d7af6b22da4469e81c05891b379afdc" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 37651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:41:52.615906 8671 consensus_queue.cc:1035] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bac1f51acec42c9b203a91a60d1d678" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:41:52.620990 8184 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:41:52.621116 8184 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:41:52.621903 8184 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 17b89f9d3d5a40ff864f83acf96e8757: Sending DeleteTablet for 3 replicas of tablet a6a90edc6662423e89d979320ccfb2dd
I20260928 14:41:52.622221 8578 tablet_service.cc:1514] Processing DeleteTablet for tablet a6a90edc6662423e89d979320ccfb2dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:52 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:47608
I20260928 14:41:52.622464 8684 tablet_replica.cc:330] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da: stopping tablet replica
I20260928 14:41:52.622566 8684 raft_consensus.cc:2229] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:41:52.622565 8310 tablet_service.cc:1514] Processing DeleteTablet for tablet a6a90edc6662423e89d979320ccfb2dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:52 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:48952
I20260928 14:41:52.622592 8444 tablet_service.cc:1514] Processing DeleteTablet for tablet a6a90edc6662423e89d979320ccfb2dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:52 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:43356
I20260928 14:41:52.622679 8684 raft_consensus.cc:2258] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:52.622781 8685 tablet_replica.cc:330] T a6a90edc6662423e89d979320ccfb2dd P 6d7af6b22da4469e81c05891b379afdc: stopping tablet replica
I20260928 14:41:52.623014 8685 raft_consensus.cc:2229] T a6a90edc6662423e89d979320ccfb2dd P 6d7af6b22da4469e81c05891b379afdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:41:52.623036 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 8247
I20260928 14:41:52.623174 8686 tablet_replica.cc:330] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678: stopping tablet replica
I20260928 14:41:52.623629 8686 raft_consensus.cc:2229] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:41:52.623798 8686 raft_consensus.cc:2258] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:52.623994 8684 ts_tablet_manager.cc:1905] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:52.624337 8686 ts_tablet_manager.cc:1905] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:52.625842 8686 ts_tablet_manager.cc:1918] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:41:52.625936 8686 log.cc:1196] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/a6a90edc6662423e89d979320ccfb2dd
I20260928 14:41:52.626237 8686 ts_tablet_manager.cc:1939] T a6a90edc6662423e89d979320ccfb2dd P 2bac1f51acec42c9b203a91a60d1d678: Deleting consensus metadata
I20260928 14:41:52.626684 8172 catalog_manager.cc:4841] TS 2bac1f51acec42c9b203a91a60d1d678 (127.22.13.66:36647): tablet a6a90edc6662423e89d979320ccfb2dd (table test-table [id=e076b7f00be44c53b5e801262965b5b0]) successfully deleted
W20260928 14:41:52.628489 8172 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:37651 (error 108)
I20260928 14:41:52.629026 8655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.628901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:37651 (local address 127.0.0.1:48958)
0928 14:41:52.628956 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:52.628998 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 14:41:52.629156 8172 catalog_manager.cc:4584] TS 6d7af6b22da4469e81c05891b379afdc (127.22.13.65:37651): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a6a90edc6662423e89d979320ccfb2dd: Network error: Client connection negotiation failed: client connection to 127.22.13.65:37651: connect: Connection refused (error 111)
I20260928 14:41:52.630041 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 8381
I20260928 14:41:52.631707 8684 ts_tablet_manager.cc:1918] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:41:52.631798 8684 log.cc:1196] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/a6a90edc6662423e89d979320ccfb2dd
I20260928 14:41:52.632071 8684 ts_tablet_manager.cc:1939] T a6a90edc6662423e89d979320ccfb2dd P e2896a5abff440f2b53412b83938c1da: Deleting consensus metadata
I20260928 14:41:52.632762 8172 catalog_manager.cc:4841] TS e2896a5abff440f2b53412b83938c1da (127.22.13.67:43103): tablet a6a90edc6662423e89d979320ccfb2dd (table test-table [id=e076b7f00be44c53b5e801262965b5b0]) successfully deleted
I20260928 14:41:52.635990 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 8515
I20260928 14:41:52.641116 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 8153
2026-09-28T14:41:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1918 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-28T14:41:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:41:52Z Disabled control of system clock
I20260928 14:41:52.669366 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:39325
--webserver_interface=127.22.13.126
--webserver_port=0
--builtin_ntp_servers=127.22.13.84:44181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:39325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:41:52.751766 8691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:52.752032 8691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:52.752058 8691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:52.753315 8691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:41:52.753372 8691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:52.753386 8691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:52.753400 8691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:41:52.753412 8691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:41:52.754936 8691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:39325
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:39325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:52.755172 8691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:52.755438 8691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:52.760383 8699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:52.760402 8696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:52.760524 8697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:52.760834 8691 server_base.cc:1034] running on GCE node
I20260928 14:41:52.761101 8691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:52.761409 8691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:52.762576 8691 hybrid_clock.cc:648] HybridClock initialized: now 1790606512762553 us; error 45 us; skew 500 ppm
I20260928 14:41:52.763943 8691 webserver.cc:460] Webserver started at http://127.22.13.126:38837/ using document root <none> and password file <none>
I20260928 14:41:52.764209 8691 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:52.764284 8691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:52.764441 8691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:52.765556 8691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "f2061e019d774568b03e135b3f4d1291"
format_stamp: "Formatted at 2026-09-28 14:41:52 on dist-test-slave-wwwb"
server_key: "057c3c48967e3c0f7bfa0324faac57db"
server_key_iv: "8cef04583749030b2842c039862b61b4"
server_key_version: "encryptionkey@0"
I20260928 14:41:52.766018 8691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "f2061e019d774568b03e135b3f4d1291"
format_stamp: "Formatted at 2026-09-28 14:41:52 on dist-test-slave-wwwb"
server_key: "057c3c48967e3c0f7bfa0324faac57db"
server_key_iv: "8cef04583749030b2842c039862b61b4"
server_key_version: "encryptionkey@0"
I20260928 14:41:52.767452 8691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:52.768275 8705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:52.768471 8691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:52.768573 8691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "f2061e019d774568b03e135b3f4d1291"
format_stamp: "Formatted at 2026-09-28 14:41:52 on dist-test-slave-wwwb"
server_key: "057c3c48967e3c0f7bfa0324faac57db"
server_key_iv: "8cef04583749030b2842c039862b61b4"
server_key_version: "encryptionkey@0"
I20260928 14:41:52.768662 8691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:52.787190 8691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:52.787570 8691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:52.787726 8691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:52.792060 8691 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:39325
I20260928 14:41:52.792150 8757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:39325 every 8 connection(s)
I20260928 14:41:52.792502 8691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:41:52.793634 8758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:52.794040 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 8691
I20260928 14:41:52.794175 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:41:52.794397 22581 external_mini_cluster.cc:1428] Setting key 2f561662bc54162551d0290ed0867df1
I20260928 14:41:52.797046 8758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291: Bootstrap starting.
I20260928 14:41:52.797844 8758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:52.798163 8758 log.cc:824] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:52.798949 8758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291: No bootstrap required, opened a new log
I20260928 14:41:52.800498 8761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.794933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49264 (local address 127.22.13.126:39325)
0928 14:41:52.795170 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:52.795180 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:52.795217 (+ 37us) server_negotiation.cc:407] Connection header received
0928 14:41:52.795732 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:52.795741 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:52.795828 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:52.795984 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:52.796154 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.797493 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.798381 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.798937 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.799086 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.799781 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:52.799785 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:52.799785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:52.799995 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:52.800050 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:52.800067 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:52.800138 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:52.800199 (+ 61us) server_negotiation.cc:299] Negotiation successful
0928 14:41:52.800289 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
I20260928 14:41:52.801349 8758 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2061e019d774568b03e135b3f4d1291" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39325 } }
I20260928 14:41:52.801484 8758 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:52.801506 8758 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2061e019d774568b03e135b3f4d1291, State: Initialized, Role: FOLLOWER
I20260928 14:41:52.801599 8758 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2061e019d774568b03e135b3f4d1291" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39325 } }
I20260928 14:41:52.801648 8758 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:41:52.801669 8758 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:41:52.801697 8758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:52.802312 8758 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2061e019d774568b03e135b3f4d1291" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39325 } }
I20260928 14:41:52.802405 8758 leader_election.cc:302] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2061e019d774568b03e135b3f4d1291; no voters:
I20260928 14:41:52.802547 8758 leader_election.cc:288] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:41:52.802624 8763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:41:52.802793 8763 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [term 1 LEADER]: Becoming Leader. State: Replica: f2061e019d774568b03e135b3f4d1291, State: Running, Role: LEADER
I20260928 14:41:52.802875 8758 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:41:52.802888 8763 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2061e019d774568b03e135b3f4d1291" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39325 } }
I20260928 14:41:52.803310 8765 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2061e019d774568b03e135b3f4d1291. Latest consensus state: current_term: 1 leader_uuid: "f2061e019d774568b03e135b3f4d1291" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2061e019d774568b03e135b3f4d1291" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39325 } } }
I20260928 14:41:52.803333 8764 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2061e019d774568b03e135b3f4d1291" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2061e019d774568b03e135b3f4d1291" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39325 } } }
I20260928 14:41:52.803468 8764 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:52.803939 8765 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:52.804116 8772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:41:52.804636 8772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:41:52.806270 8772 catalog_manager.cc:1347] Generated new cluster ID: 3856b245911d430b8e0c1c2f0388d315
I20260928 14:41:52.806334 8772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:41:52.827595 8772 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:41:52.828329 8772 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:41:52.836076 8772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291: Generated new TSK 0
I20260928 14:41:52.836349 8772 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:41:52.847294 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:39325
--builtin_ntp_servers=127.22.13.84:44181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:41:52.931845 8782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:52.932107 8782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:52.932133 8782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:52.933403 8782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:52.933451 8782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:52.933478 8782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:41:52.935038 8782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:39325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:52.935343 8782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:52.935659 8782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:52.940801 8787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:52.940824 8788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:52.940801 8790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:52.941080 8782 server_base.cc:1034] running on GCE node
I20260928 14:41:52.941282 8782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:52.941499 8782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:52.942684 8782 hybrid_clock.cc:648] HybridClock initialized: now 1790606512942670 us; error 31 us; skew 500 ppm
I20260928 14:41:52.943986 8782 webserver.cc:460] Webserver started at http://127.22.13.65:41675/ using document root <none> and password file <none>
I20260928 14:41:52.944212 8782 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:52.944255 8782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:52.944350 8782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:52.945401 8782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "d09b2b02c2f94d018ac7173eeb8d883e"
format_stamp: "Formatted at 2026-09-28 14:41:52 on dist-test-slave-wwwb"
server_key: "3dbe396129ce0f22477b24dd48ae5442"
server_key_iv: "e41558be18feee0e7ea7db4416a5154a"
server_key_version: "encryptionkey@0"
I20260928 14:41:52.945753 8782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "d09b2b02c2f94d018ac7173eeb8d883e"
format_stamp: "Formatted at 2026-09-28 14:41:52 on dist-test-slave-wwwb"
server_key: "3dbe396129ce0f22477b24dd48ae5442"
server_key_iv: "e41558be18feee0e7ea7db4416a5154a"
server_key_version: "encryptionkey@0"
I20260928 14:41:52.947139 8782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:41:52.947881 8796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:52.948038 8782 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:52.948128 8782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "d09b2b02c2f94d018ac7173eeb8d883e"
format_stamp: "Formatted at 2026-09-28 14:41:52 on dist-test-slave-wwwb"
server_key: "3dbe396129ce0f22477b24dd48ae5442"
server_key_iv: "e41558be18feee0e7ea7db4416a5154a"
server_key_version: "encryptionkey@0"
I20260928 14:41:52.948187 8782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:52.973337 8782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:52.973639 8782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:52.973748 8782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:52.974018 8782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:52.974334 8782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:52.974366 8782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:52.974390 8782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:52.974408 8782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:52.980122 8782 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:33191
I20260928 14:41:52.980221 8909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:33191 every 8 connection(s)
I20260928 14:41:52.980593 8782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:41:52.982668 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 8782
I20260928 14:41:52.982766 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:41:52.982954 22581 external_mini_cluster.cc:1428] Setting key 1794134b03e425086d510ef762847e68
I20260928 14:41:52.983978 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:39325
--builtin_ntp_servers=127.22.13.84:44181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 14:41:52.987215 8761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.981429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:58479 (local address 127.22.13.126:39325)
0928 14:41:52.981502 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:52.981504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:52.981783 (+ 279us) server_negotiation.cc:407] Connection header received
0928 14:41:52.982351 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:52.982352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:52.982362 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:52.982416 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:52.983214 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.983875 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.985479 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.985831 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.985916 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.986876 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:52.986877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:52.986878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:52.986884 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:52.986902 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:52.986907 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:52.986967 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:52.987057 (+ 90us) server_negotiation.cc:299] Negotiation successful
0928 14:41:52.987092 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 14:41:52.987336 8912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:52.981412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39325 (local address 127.22.13.65:58479)
0928 14:41:52.981635 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:52.981671 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:52.982255 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:52.982489 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:52.982493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:52.982501 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:52.983132 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.983137 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.983980 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.983982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.985366 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:52.985371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:52.985930 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:52.985931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:52.986109 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:52.986727 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:41:52.986728 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:52.986759 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:41:52.986786 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:52.986978 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:52.986983 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:52.986986 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:52.987040 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 14:41:52.987133 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":115,"threads_started":1}
I20260928 14:41:52.987949 8910 heartbeater.cc:347] Connected to a master server at 127.22.13.126:39325
I20260928 14:41:52.988062 8910 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:52.988277 8910 heartbeater.cc:511] Master 127.22.13.126:39325 requested a full tablet report, sending...
I20260928 14:41:52.988857 8722 ts_manager.cc:194] Registered new tserver with Master: d09b2b02c2f94d018ac7173eeb8d883e (127.22.13.65:33191)
I20260928 14:41:52.989905 8722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.13.65:58479
W20260928 14:41:53.066799 8913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:53.067026 8913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:53.067057 8913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:53.068198 8913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:53.068239 8913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:53.068265 8913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:41:53.069615 8913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:39325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:53.069840 8913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:53.070132 8913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:53.075410 8919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:53.075445 8918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:53.075534 8913 server_base.cc:1034] running on GCE node
W20260928 14:41:53.075687 8921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:53.075932 8913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:53.076193 8913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:53.077359 8913 hybrid_clock.cc:648] HybridClock initialized: now 1790606513077342 us; error 30 us; skew 500 ppm
I20260928 14:41:53.078684 8913 webserver.cc:460] Webserver started at http://127.22.13.66:44795/ using document root <none> and password file <none>
I20260928 14:41:53.078945 8913 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:53.079017 8913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:53.079181 8913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:53.080298 8913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "09e095f8244c44ecae56b9fc1bbcef58"
format_stamp: "Formatted at 2026-09-28 14:41:53 on dist-test-slave-wwwb"
server_key: "9a43a09e2ec5bf5b2c90fd2f74b91963"
server_key_iv: "0ccf092acb9b0acdff602645d65108db"
server_key_version: "encryptionkey@0"
I20260928 14:41:53.080684 8913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "09e095f8244c44ecae56b9fc1bbcef58"
format_stamp: "Formatted at 2026-09-28 14:41:53 on dist-test-slave-wwwb"
server_key: "9a43a09e2ec5bf5b2c90fd2f74b91963"
server_key_iv: "0ccf092acb9b0acdff602645d65108db"
server_key_version: "encryptionkey@0"
I20260928 14:41:53.082494 8913 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260928 14:41:53.083276 8927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:53.083452 8913 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:53.083559 8913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "09e095f8244c44ecae56b9fc1bbcef58"
format_stamp: "Formatted at 2026-09-28 14:41:53 on dist-test-slave-wwwb"
server_key: "9a43a09e2ec5bf5b2c90fd2f74b91963"
server_key_iv: "0ccf092acb9b0acdff602645d65108db"
server_key_version: "encryptionkey@0"
I20260928 14:41:53.083650 8913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:53.097280 8913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:53.097651 8913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:53.097810 8913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:53.098114 8913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:53.098467 8913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:53.098524 8913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:53.098560 8913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:53.098588 8913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:53.104982 8913 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:46011
I20260928 14:41:53.105039 9040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:46011 every 8 connection(s)
I20260928 14:41:53.105401 8913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:41:53.110149 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 8913
I20260928 14:41:53.110273 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:41:53.110482 22581 external_mini_cluster.cc:1428] Setting key b0698ab404ef957106bad7055e933349
I20260928 14:41:53.111228 8761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.106244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:43551 (local address 127.22.13.126:39325)
0928 14:41:53.106310 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:53.106312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:53.106687 (+ 375us) server_negotiation.cc:407] Connection header received
0928 14:41:53.107197 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:53.107198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:53.107208 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:53.107262 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:53.107954 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.108452 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.109465 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.109653 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.109707 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:53.110890 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:53.110891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:53.110892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:53.110896 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:53.110910 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:53.110914 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:53.110960 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:53.111080 (+ 120us) server_negotiation.cc:299] Negotiation successful
0928 14:41:53.111108 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 14:41:53.111454 9043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.106275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39325 (local address 127.22.13.66:43551)
0928 14:41:53.106535 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:53.106586 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:53.107107 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:53.107319 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:53.107322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:53.107330 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:53.107871 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:53.107877 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.108550 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.108551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:53.109377 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:53.109380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.109735 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.109736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:53.109855 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:53.110638 (+ 783us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:41:53.110641 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:53.110701 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:41:53.110737 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:53.110984 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:53.110991 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:53.110994 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:53.111077 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 14:41:53.111202 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":115,"threads_started":1}
I20260928 14:41:53.111666 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:39325
--builtin_ntp_servers=127.22.13.84:44181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 14:41:53.111963 9041 heartbeater.cc:347] Connected to a master server at 127.22.13.126:39325
I20260928 14:41:53.112069 9041 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:53.112272 9041 heartbeater.cc:511] Master 127.22.13.126:39325 requested a full tablet report, sending...
I20260928 14:41:53.112671 8722 ts_manager.cc:194] Registered new tserver with Master: 09e095f8244c44ecae56b9fc1bbcef58 (127.22.13.66:46011)
I20260928 14:41:53.113243 8722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.13.66:43551
W20260928 14:41:53.194759 9044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:53.195003 9044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:53.195050 9044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:53.196249 9044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:53.196319 9044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:53.196365 9044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:41:53.197738 9044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:39325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:53.198040 9044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:53.198344 9044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:53.203464 9050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:53.203469 9049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:53.203469 9052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:53.203938 9044 server_base.cc:1034] running on GCE node
I20260928 14:41:53.204159 9044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:53.204401 9044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:53.205597 9044 hybrid_clock.cc:648] HybridClock initialized: now 1790606513205586 us; error 40 us; skew 500 ppm
I20260928 14:41:53.207044 9044 webserver.cc:460] Webserver started at http://127.22.13.67:41199/ using document root <none> and password file <none>
I20260928 14:41:53.207304 9044 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:53.207388 9044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:53.207577 9044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:53.208678 9044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "f014294c60574aae9f86c62f5f26579c"
format_stamp: "Formatted at 2026-09-28 14:41:53 on dist-test-slave-wwwb"
server_key: "7e7045b6c5cc88bb22c27f25c1563e31"
server_key_iv: "eefb3deafe024ca622f433dd44ba5b72"
server_key_version: "encryptionkey@0"
I20260928 14:41:53.209067 9044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "f014294c60574aae9f86c62f5f26579c"
format_stamp: "Formatted at 2026-09-28 14:41:53 on dist-test-slave-wwwb"
server_key: "7e7045b6c5cc88bb22c27f25c1563e31"
server_key_iv: "eefb3deafe024ca622f433dd44ba5b72"
server_key_version: "encryptionkey@0"
I20260928 14:41:53.210500 9044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:41:53.211266 9058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:53.211436 9044 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:53.211548 9044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "f014294c60574aae9f86c62f5f26579c"
format_stamp: "Formatted at 2026-09-28 14:41:53 on dist-test-slave-wwwb"
server_key: "7e7045b6c5cc88bb22c27f25c1563e31"
server_key_iv: "eefb3deafe024ca622f433dd44ba5b72"
server_key_version: "encryptionkey@0"
I20260928 14:41:53.211628 9044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:53.232561 9044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:53.232930 9044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:53.233098 9044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:53.233354 9044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:53.233737 9044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:53.233793 9044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:53.233834 9044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:53.233877 9044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:53.239933 9044 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:41195
I20260928 14:41:53.239974 9171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:41195 every 8 connection(s)
I20260928 14:41:53.240335 9044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:41:53.245739 8761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.241199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:38869 (local address 127.22.13.126:39325)
0928 14:41:53.241263 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:53.241265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:53.241488 (+ 223us) server_negotiation.cc:407] Connection header received
0928 14:41:53.242053 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:53.242054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:53.242063 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:53.242131 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:53.242817 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.243341 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.244314 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.244539 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.244597 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:53.245492 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:53.245493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:53.245493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:53.245497 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:53.245508 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:53.245512 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:53.245553 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:53.245652 (+ 99us) server_negotiation.cc:299] Negotiation successful
0928 14:41:53.245674 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:41:53.245983 9174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.241169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39325 (local address 127.22.13.67:38869)
0928 14:41:53.241354 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:53.241396 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:53.241901 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:53.242217 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:53.242221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:53.242230 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:53.242740 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:53.242744 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.243429 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.243429 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:53.244222 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:53.244224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.244621 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.244621 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:53.244737 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:53.245358 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:41:53.245359 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:53.245390 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:41:53.245414 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:53.245579 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:53.245583 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:53.245586 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:53.245642 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 14:41:53.245730 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
I20260928 14:41:53.246583 9172 heartbeater.cc:347] Connected to a master server at 127.22.13.126:39325
I20260928 14:41:53.246701 9172 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:53.246909 9172 heartbeater.cc:511] Master 127.22.13.126:39325 requested a full tablet report, sending...
I20260928 14:41:53.247328 8722 ts_manager.cc:194] Registered new tserver with Master: f014294c60574aae9f86c62f5f26579c (127.22.13.67:41195)
I20260928 14:41:53.247463 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 9044
I20260928 14:41:53.247588 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:41:53.247821 22581 external_mini_cluster.cc:1428] Setting key 545a6f9cefe6a29108e8550feb7c141b
I20260928 14:41:53.247822 8722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.13.67:38869
I20260928 14:41:53.248708 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:41:53.252960 8761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.250130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49266 (local address 127.22.13.126:39325)
0928 14:41:53.250172 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:53.250174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:53.250307 (+ 133us) server_negotiation.cc:407] Connection header received
0928 14:41:53.250338 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:53.250339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:53.250348 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:53.250419 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:53.250567 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.251009 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.251758 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.251932 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.251986 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:53.252742 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:53.252743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:53.252743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:53.252747 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:53.252758 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:53.252762 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:53.252825 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:53.252892 (+ 67us) server_negotiation.cc:299] Negotiation successful
0928 14:41:53.252902 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:41:53.254696 8722 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:41:53.256222 8722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:41:53.262657 9181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.258511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:46011 (local address 127.0.0.1:53078)
0928 14:41:53.258676 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:53.258718 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:53.258786 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:53.259157 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:53.259160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:53.259162 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:53.259237 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:53.259239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.260331 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.260332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:53.261382 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:53.261384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.262336 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.262337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:53.262417 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:53.262432 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 14:41:53.262509 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:41:53.262571 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":62,"threads_started":1}
I20260928 14:41:53.262660 9184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.258843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53078 (local address 127.22.13.66:46011)
0928 14:41:53.258990 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:53.258994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:53.259002 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:41:53.259034 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:53.259036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:53.259049 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:53.259153 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:53.259296 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.260239 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.261481 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.262250 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.262322 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:53.262410 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:53.262536 (+ 126us) server_negotiation.cc:299] Negotiation successful
0928 14:41:53.262575 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260928 14:41:53.263121 9185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.258985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50834 (local address 127.22.13.65:33191)
0928 14:41:53.259156 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:53.259160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:53.259563 (+ 403us) server_negotiation.cc:407] Connection header received
0928 14:41:53.259648 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:53.259650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:53.259666 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:53.259745 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:53.259864 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.260894 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.262072 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.262772 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.262842 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:53.262925 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:53.262975 (+ 50us) server_negotiation.cc:299] Negotiation successful
0928 14:41:53.263011 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260928 14:41:53.263118 9180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.258710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:33191 (local address 127.0.0.1:50834)
0928 14:41:53.259488 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:53.259499 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:53.259558 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:53.259756 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:53.259757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:53.259759 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:53.259797 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:53.259799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.260979 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.260980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:53.261999 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:53.262002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.262842 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.262843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:53.262902 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:53.262905 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:53.262961 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 14:41:53.263005 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"thread_start_us":66,"threads_started":1}
I20260928 14:41:53.263257 9183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.258511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:41195 (local address 127.0.0.1:55200)
0928 14:41:53.258820 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:53.258829 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:53.258869 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:53.259396 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:53.259397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:53.259399 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:53.259448 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:53.259450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.260880 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.260881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:53.261896 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:53.261899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.262915 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.262916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:53.262979 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:53.262984 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:41:53.263037 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 14:41:53.263107 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
I20260928 14:41:53.263514 9182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.258622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55200 (local address 127.22.13.67:41195)
0928 14:41:53.259239 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:53.259242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:53.259253 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:41:53.259296 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:53.259298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:53.259313 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:53.259619 (+ 306us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:53.259682 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.260790 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.262011 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.262640 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.262702 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:53.263377 (+ 675us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:53.263404 (+ 27us) server_negotiation.cc:299] Negotiation successful
0928 14:41:53.263443 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":92,"threads_started":1}
I20260928 14:41:53.263852 8975 tablet_service.cc:1467] Processing CreateTablet for tablet 4492a1518a4b41349fbb7cd4323f7ece (DEFAULT_TABLE table=test-table [id=e7f13c6312d24bdda45f503ab371f256]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:53.264165 8975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4492a1518a4b41349fbb7cd4323f7ece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:53.266712 8844 tablet_service.cc:1467] Processing CreateTablet for tablet 4492a1518a4b41349fbb7cd4323f7ece (DEFAULT_TABLE table=test-table [id=e7f13c6312d24bdda45f503ab371f256]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:53.266812 9106 tablet_service.cc:1467] Processing CreateTablet for tablet 4492a1518a4b41349fbb7cd4323f7ece (DEFAULT_TABLE table=test-table [id=e7f13c6312d24bdda45f503ab371f256]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:53.267076 8844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4492a1518a4b41349fbb7cd4323f7ece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:53.267185 9106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4492a1518a4b41349fbb7cd4323f7ece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:53.267362 9186 tablet_bootstrap.cc:492] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58: Bootstrap starting.
I20260928 14:41:53.268020 9186 tablet_bootstrap.cc:654] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:53.268353 9186 log.cc:824] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:53.269534 9186 tablet_bootstrap.cc:492] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58: No bootstrap required, opened a new log
I20260928 14:41:53.269616 9186 ts_tablet_manager.cc:1397] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:41:53.270221 9188 tablet_bootstrap.cc:492] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e: Bootstrap starting.
I20260928 14:41:53.270931 9188 tablet_bootstrap.cc:654] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:53.271292 9188 log.cc:824] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:53.272089 9188 tablet_bootstrap.cc:492] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e: No bootstrap required, opened a new log
I20260928 14:41:53.272188 9188 ts_tablet_manager.cc:1397] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:41:53.272476 9186 raft_consensus.cc:348] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f014294c60574aae9f86c62f5f26579c" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41195 } } peers { permanent_uuid: "d09b2b02c2f94d018ac7173eeb8d883e" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 33191 } } peers { permanent_uuid: "09e095f8244c44ecae56b9fc1bbcef58" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 46011 } }
I20260928 14:41:53.272684 9186 raft_consensus.cc:374] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:53.272724 9186 raft_consensus.cc:729] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09e095f8244c44ecae56b9fc1bbcef58, State: Initialized, Role: FOLLOWER
I20260928 14:41:53.272886 9186 consensus_queue.cc:260] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f014294c60574aae9f86c62f5f26579c" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41195 } } peers { permanent_uuid: "d09b2b02c2f94d018ac7173eeb8d883e" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 33191 } } peers { permanent_uuid: "09e095f8244c44ecae56b9fc1bbcef58" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 46011 } }
I20260928 14:41:53.273175 9186 ts_tablet_manager.cc:1428] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260928 14:41:53.273237 9041 heartbeater.cc:503] Master 127.22.13.126:39325 was elected leader, sending a full tablet report...
I20260928 14:41:53.273433 9189 tablet_bootstrap.cc:492] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c: Bootstrap starting.
I20260928 14:41:53.274176 9189 tablet_bootstrap.cc:654] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:53.274550 9189 log.cc:824] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:53.274740 9188 raft_consensus.cc:348] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f014294c60574aae9f86c62f5f26579c" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41195 } } peers { permanent_uuid: "d09b2b02c2f94d018ac7173eeb8d883e" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 33191 } } peers { permanent_uuid: "09e095f8244c44ecae56b9fc1bbcef58" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 46011 } }
I20260928 14:41:53.274871 9188 raft_consensus.cc:374] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:53.274892 9188 raft_consensus.cc:729] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d09b2b02c2f94d018ac7173eeb8d883e, State: Initialized, Role: FOLLOWER
I20260928 14:41:53.275010 9188 consensus_queue.cc:260] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f014294c60574aae9f86c62f5f26579c" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41195 } } peers { permanent_uuid: "d09b2b02c2f94d018ac7173eeb8d883e" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 33191 } } peers { permanent_uuid: "09e095f8244c44ecae56b9fc1bbcef58" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 46011 } }
I20260928 14:41:53.275259 9188 ts_tablet_manager.cc:1428] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:41:53.275306 8910 heartbeater.cc:503] Master 127.22.13.126:39325 was elected leader, sending a full tablet report...
I20260928 14:41:53.275367 9189 tablet_bootstrap.cc:492] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c: No bootstrap required, opened a new log
I20260928 14:41:53.275430 9189 ts_tablet_manager.cc:1397] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:41:53.277869 9189 raft_consensus.cc:348] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f014294c60574aae9f86c62f5f26579c" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41195 } } peers { permanent_uuid: "d09b2b02c2f94d018ac7173eeb8d883e" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 33191 } } peers { permanent_uuid: "09e095f8244c44ecae56b9fc1bbcef58" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 46011 } }
I20260928 14:41:53.278105 9189 raft_consensus.cc:374] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:53.278132 9189 raft_consensus.cc:729] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f014294c60574aae9f86c62f5f26579c, State: Initialized, Role: FOLLOWER
I20260928 14:41:53.278218 9189 consensus_queue.cc:260] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f014294c60574aae9f86c62f5f26579c" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41195 } } peers { permanent_uuid: "d09b2b02c2f94d018ac7173eeb8d883e" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 33191 } } peers { permanent_uuid: "09e095f8244c44ecae56b9fc1bbcef58" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 46011 } }
I20260928 14:41:53.278448 9189 ts_tablet_manager.cc:1428] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:41:53.278529 9172 heartbeater.cc:503] Master 127.22.13.126:39325 was elected leader, sending a full tablet report...
I20260928 14:41:53.343864 9194 raft_consensus.cc:482] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:41:53.343997 9194 raft_consensus.cc:504] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f014294c60574aae9f86c62f5f26579c" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41195 } } peers { permanent_uuid: "d09b2b02c2f94d018ac7173eeb8d883e" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 33191 } } peers { permanent_uuid: "09e095f8244c44ecae56b9fc1bbcef58" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 46011 } }
I20260928 14:41:53.344365 9194 leader_election.cc:288] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d09b2b02c2f94d018ac7173eeb8d883e (127.22.13.65:33191), 09e095f8244c44ecae56b9fc1bbcef58 (127.22.13.66:46011)
I20260928 14:41:53.347209 9185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.344656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:48501 (local address 127.22.13.65:33191)
0928 14:41:53.344725 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:53.344727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:53.344734 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:41:53.344768 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:53.344769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:53.344777 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:53.344844 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:53.345003 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.345474 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.346480 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.346838 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.346922 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:53.346931 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:53.347083 (+ 152us) server_negotiation.cc:299] Negotiation successful
0928 14:41:53.347139 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:41:53.347209 9174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.344549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:33191 (local address 127.22.13.67:48501)
0928 14:41:53.344610 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:53.344623 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:53.344668 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:53.344849 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:53.344850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:53.344851 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:53.344929 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:53.344930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.345564 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.345565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:53.346372 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:53.346374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.346920 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.346921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:53.346995 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:53.346998 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:53.347076 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:41:53.347131 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260928 14:41:53.347461 9195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.344564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:46011 (local address 127.22.13.67:55081)
0928 14:41:53.344781 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:53.344789 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:53.344867 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:53.345016 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:53.345017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:53.345018 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:53.345095 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:53.345096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.345727 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.345728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:53.346577 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:53.346579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.347153 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.347154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:53.347238 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:53.347241 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:53.347307 (+ 66us) client_negotiation.cc:240] Negotiation successful
0928 14:41:53.347354 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":113,"threads_started":1}
I20260928 14:41:53.347587 9184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.344645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:55081 (local address 127.22.13.66:46011)
0928 14:41:53.344694 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:53.344696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:53.344859 (+ 163us) server_negotiation.cc:407] Connection header received
0928 14:41:53.344933 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:53.344934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:53.344952 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:53.345014 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:53.345165 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.345637 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.346680 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.347056 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.347153 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:53.347161 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:53.347440 (+ 279us) server_negotiation.cc:299] Negotiation successful
0928 14:41:53.347510 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:41:53.347663 8864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4492a1518a4b41349fbb7cd4323f7ece" candidate_uuid: "f014294c60574aae9f86c62f5f26579c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d09b2b02c2f94d018ac7173eeb8d883e" is_pre_election: true
I20260928 14:41:53.347841 8864 raft_consensus.cc:2454] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f014294c60574aae9f86c62f5f26579c in term 0.
I20260928 14:41:53.347847 8995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4492a1518a4b41349fbb7cd4323f7ece" candidate_uuid: "f014294c60574aae9f86c62f5f26579c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09e095f8244c44ecae56b9fc1bbcef58" is_pre_election: true
I20260928 14:41:53.347965 8995 raft_consensus.cc:2454] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f014294c60574aae9f86c62f5f26579c in term 0.
I20260928 14:41:53.348112 9060 leader_election.cc:302] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d09b2b02c2f94d018ac7173eeb8d883e, f014294c60574aae9f86c62f5f26579c; no voters:
I20260928 14:41:53.348244 9194 raft_consensus.cc:2781] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:41:53.348300 9194 raft_consensus.cc:482] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:41:53.348323 9194 raft_consensus.cc:3037] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:53.349154 9194 raft_consensus.cc:504] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f014294c60574aae9f86c62f5f26579c" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41195 } } peers { permanent_uuid: "d09b2b02c2f94d018ac7173eeb8d883e" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 33191 } } peers { permanent_uuid: "09e095f8244c44ecae56b9fc1bbcef58" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 46011 } }
I20260928 14:41:53.349327 9194 leader_election.cc:288] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [CANDIDATE]: Term 1 election: Requested vote from peers d09b2b02c2f94d018ac7173eeb8d883e (127.22.13.65:33191), 09e095f8244c44ecae56b9fc1bbcef58 (127.22.13.66:46011)
I20260928 14:41:53.349462 8864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4492a1518a4b41349fbb7cd4323f7ece" candidate_uuid: "f014294c60574aae9f86c62f5f26579c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d09b2b02c2f94d018ac7173eeb8d883e"
I20260928 14:41:53.349498 8995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4492a1518a4b41349fbb7cd4323f7ece" candidate_uuid: "f014294c60574aae9f86c62f5f26579c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09e095f8244c44ecae56b9fc1bbcef58"
I20260928 14:41:53.349563 8864 raft_consensus.cc:3037] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:53.349573 8995 raft_consensus.cc:3037] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:53.350445 8864 raft_consensus.cc:2454] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f014294c60574aae9f86c62f5f26579c in term 1.
I20260928 14:41:53.350453 8995 raft_consensus.cc:2454] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f014294c60574aae9f86c62f5f26579c in term 1.
I20260928 14:41:53.350653 9062 leader_election.cc:302] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09e095f8244c44ecae56b9fc1bbcef58, f014294c60574aae9f86c62f5f26579c; no voters:
I20260928 14:41:53.350751 9194 raft_consensus.cc:2781] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:41:53.350893 9194 raft_consensus.cc:686] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [term 1 LEADER]: Becoming Leader. State: Replica: f014294c60574aae9f86c62f5f26579c, State: Running, Role: LEADER
I20260928 14:41:53.350999 9194 consensus_queue.cc:237] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f014294c60574aae9f86c62f5f26579c" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41195 } } peers { permanent_uuid: "d09b2b02c2f94d018ac7173eeb8d883e" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 33191 } } peers { permanent_uuid: "09e095f8244c44ecae56b9fc1bbcef58" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 46011 } }
I20260928 14:41:53.351895 8722 catalog_manager.cc:5495] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c reported cstate change: term changed from 0 to 1, leader changed from <none> to f014294c60574aae9f86c62f5f26579c (127.22.13.67). New cstate: current_term: 1 leader_uuid: "f014294c60574aae9f86c62f5f26579c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f014294c60574aae9f86c62f5f26579c" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 41195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d09b2b02c2f94d018ac7173eeb8d883e" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 33191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09e095f8244c44ecae56b9fc1bbcef58" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 46011 } health_report { overall_health: UNKNOWN } } }
W20260928 14:41:53.355970 9042 tablet.cc:2369] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:41:53.357517 9182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.355190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55214 (local address 127.22.13.67:41195)
0928 14:41:53.355253 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:53.355255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:53.355263 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:41:53.355297 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:53.355298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:53.355320 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:53.355384 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:41:53.355493 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.355949 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.356753 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:53.356936 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:53.356990 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:53.357123 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:41:53.357208 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:41:53.357277 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:53.357393 (+ 116us) server_negotiation.cc:299] Negotiation successful
0928 14:41:53.357404 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:41:53.359330 8864 raft_consensus.cc:1261] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e [term 1 FOLLOWER]: Refusing update from remote peer f014294c60574aae9f86c62f5f26579c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:41:53.359622 8995 raft_consensus.cc:1261] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58 [term 1 FOLLOWER]: Refusing update from remote peer f014294c60574aae9f86c62f5f26579c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:41:53.359668 9196 consensus_queue.cc:1035] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d09b2b02c2f94d018ac7173eeb8d883e" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 33191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:41:53.359913 9196 consensus_queue.cc:1035] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [LEADER]: Connected to new peer: Peer: permanent_uuid: "09e095f8244c44ecae56b9fc1bbcef58" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 46011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:41:53.362919 9201 mvcc.cc:205] Tried to move back new op lower bound from 7334324278716891136 to 7334324278686425088. Current Snapshot: MvccSnapshot[applied={T|T < 7334324278716891136}]
I20260928 14:41:53.365139 8722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:41:53.365248 8722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:41:53.366070 8722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f2061e019d774568b03e135b3f4d1291: Sending DeleteTablet for 3 replicas of tablet 4492a1518a4b41349fbb7cd4323f7ece
I20260928 14:41:53.366446 9106 tablet_service.cc:1514] Processing DeleteTablet for tablet 4492a1518a4b41349fbb7cd4323f7ece with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:53 UTC) from {username='slave'} at 127.0.0.1:55200
I20260928 14:41:53.366472 8975 tablet_service.cc:1514] Processing DeleteTablet for tablet 4492a1518a4b41349fbb7cd4323f7ece with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:53 UTC) from {username='slave'} at 127.0.0.1:53078
I20260928 14:41:53.366464 8844 tablet_service.cc:1514] Processing DeleteTablet for tablet 4492a1518a4b41349fbb7cd4323f7ece with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:53 UTC) from {username='slave'} at 127.0.0.1:50834
I20260928 14:41:53.366683 9209 tablet_replica.cc:330] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c: stopping tablet replica
I20260928 14:41:53.366777 9209 raft_consensus.cc:2229] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:41:53.366813 9211 tablet_replica.cc:330] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e: stopping tablet replica
I20260928 14:41:53.366901 9209 raft_consensus.cc:2258] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:53.366947 9211 raft_consensus.cc:2229] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:41:53.367092 9211 raft_consensus.cc:2258] T 4492a1518a4b41349fbb7cd4323f7ece P d09b2b02c2f94d018ac7173eeb8d883e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:53.367082 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 8782
I20260928 14:41:53.367801 9210 tablet_replica.cc:330] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58: stopping tablet replica
I20260928 14:41:53.367980 9210 raft_consensus.cc:2229] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:41:53.368110 9210 raft_consensus.cc:2258] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:53.368664 9210 ts_tablet_manager.cc:1905] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:53.369042 9209 ts_tablet_manager.cc:1905] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:53.370524 9210 ts_tablet_manager.cc:1918] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:41:53.370611 9210 log.cc:1196] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/4492a1518a4b41349fbb7cd4323f7ece
I20260928 14:41:53.370759 9209 ts_tablet_manager.cc:1918] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:41:53.370851 9209 log.cc:1196] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/4492a1518a4b41349fbb7cd4323f7ece
I20260928 14:41:53.370880 9210 ts_tablet_manager.cc:1939] T 4492a1518a4b41349fbb7cd4323f7ece P 09e095f8244c44ecae56b9fc1bbcef58: Deleting consensus metadata
I20260928 14:41:53.371125 9209 ts_tablet_manager.cc:1939] T 4492a1518a4b41349fbb7cd4323f7ece P f014294c60574aae9f86c62f5f26579c: Deleting consensus metadata
I20260928 14:41:53.371337 8709 catalog_manager.cc:4841] TS 09e095f8244c44ecae56b9fc1bbcef58 (127.22.13.66:46011): tablet 4492a1518a4b41349fbb7cd4323f7ece (table test-table [id=e7f13c6312d24bdda45f503ab371f256]) successfully deleted
I20260928 14:41:53.371555 8707 catalog_manager.cc:4841] TS f014294c60574aae9f86c62f5f26579c (127.22.13.67:41195): tablet 4492a1518a4b41349fbb7cd4323f7ece (table test-table [id=e7f13c6312d24bdda45f503ab371f256]) successfully deleted
W20260928 14:41:53.371919 8707 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:33191 (error 108)
I20260928 14:41:53.372373 9181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:53.372235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:33191 (local address 127.0.0.1:50848)
0928 14:41:53.372297 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:53.372337 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260928 14:41:53.372498 8707 catalog_manager.cc:4584] TS d09b2b02c2f94d018ac7173eeb8d883e (127.22.13.65:33191): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4492a1518a4b41349fbb7cd4323f7ece: Network error: Client connection negotiation failed: client connection to 127.22.13.65:33191: connect: Connection refused (error 111)
I20260928 14:41:53.373119 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 8913
I20260928 14:41:53.378569 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 9044
I20260928 14:41:53.383220 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 8691
2026-09-28T14:41:53Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (740 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:41:53 dist-test-slave-wwwb krb5kdc[9215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:41:53 dist-test-slave-wwwb krb5kdc[9215](info): set up 2 sockets
Sep 28 14:41:53 dist-test-slave-wwwb krb5kdc[9215](info): commencing operation
krb5kdc: starting...
W20260928 14:41:54.406090 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:41:54 dist-test-slave-wwwb krb5kdc[9215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:41:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:41:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:41:54.534907 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:35185
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:39213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:35185
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:54.617794 9228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:54.618125 9228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:54.618172 9228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:54.619422 9228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:41:54.619498 9228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:54.619520 9228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:54.619541 9228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:41:54.619567 9228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:41:54.621156 9228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:39213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:35185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:35185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:54.621459 9228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:54.621706 9228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:54.627084 9233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:54.627125 9234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:54.627084 9236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:54.627286 9228 server_base.cc:1034] running on GCE node
I20260928 14:41:54.627597 9228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:54.627936 9228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:54.629120 9228 hybrid_clock.cc:648] HybridClock initialized: now 1790606514629100 us; error 37 us; skew 500 ppm
Sep 28 14:41:54 dist-test-slave-wwwb krb5kdc[9215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606514, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:54.631855 9228 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:41:54.632321 9228 webserver.cc:460] Webserver started at http://127.22.13.126:45461/ using document root <none> and password file <none>
I20260928 14:41:54.632570 9228 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:54.632652 9228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:54.632800 9228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:54.633911 9228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "52827539fa5a41f3a4498ea402b9f639"
format_stamp: "Formatted at 2026-09-28 14:41:54 on dist-test-slave-wwwb"
server_key: "50851fa144c3c209c499cd7fd1d6696b"
server_key_iv: "b8113ea82940ecdc80385222666154c0"
server_key_version: "encryptionkey@0"
I20260928 14:41:54.634374 9228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "52827539fa5a41f3a4498ea402b9f639"
format_stamp: "Formatted at 2026-09-28 14:41:54 on dist-test-slave-wwwb"
server_key: "50851fa144c3c209c499cd7fd1d6696b"
server_key_iv: "b8113ea82940ecdc80385222666154c0"
server_key_version: "encryptionkey@0"
I20260928 14:41:54.635754 9228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:41:54.636485 9243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:54.636672 9228 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:54.636785 9228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "52827539fa5a41f3a4498ea402b9f639"
format_stamp: "Formatted at 2026-09-28 14:41:54 on dist-test-slave-wwwb"
server_key: "50851fa144c3c209c499cd7fd1d6696b"
server_key_iv: "b8113ea82940ecdc80385222666154c0"
server_key_version: "encryptionkey@0"
I20260928 14:41:54.636869 9228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:54.655441 9228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:54.658072 9228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:54.658255 9228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:54.662235 9228 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:35185
I20260928 14:41:54.662290 9295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:35185 every 8 connection(s)
I20260928 14:41:54.662698 9228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:41:54.663635 9296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:54.666088 9296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639: Bootstrap starting.
I20260928 14:41:54.666666 9296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:54.666918 9296 log.cc:824] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:54.667631 9296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639: No bootstrap required, opened a new log
I20260928 14:41:54.669265 9296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52827539fa5a41f3a4498ea402b9f639" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35185 } }
I20260928 14:41:54.669421 9296 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:54.669459 9296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52827539fa5a41f3a4498ea402b9f639, State: Initialized, Role: FOLLOWER
I20260928 14:41:54.669587 9296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52827539fa5a41f3a4498ea402b9f639" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35185 } }
I20260928 14:41:54.669641 9296 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:41:54.669662 9296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:41:54.669691 9296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:54.670248 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 9228
I20260928 14:41:54.670310 9296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52827539fa5a41f3a4498ea402b9f639" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35185 } }
I20260928 14:41:54.670384 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:41:54.670454 9296 leader_election.cc:302] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52827539fa5a41f3a4498ea402b9f639; no voters:
I20260928 14:41:54.670612 22581 external_mini_cluster.cc:1428] Setting key 7aaf358b6ee9e823eeb3e755fbfc4341
I20260928 14:41:54.670686 9296 leader_election.cc:288] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:41:54.670710 9299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:41:54.670821 9299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [term 1 LEADER]: Becoming Leader. State: Replica: 52827539fa5a41f3a4498ea402b9f639, State: Running, Role: LEADER
I20260928 14:41:54.670909 9299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52827539fa5a41f3a4498ea402b9f639" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35185 } }
I20260928 14:41:54.670970 9296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:41:54.671398 9300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52827539fa5a41f3a4498ea402b9f639" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52827539fa5a41f3a4498ea402b9f639" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35185 } } }
I20260928 14:41:54.671489 9300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:54.671650 9301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52827539fa5a41f3a4498ea402b9f639. Latest consensus state: current_term: 1 leader_uuid: "52827539fa5a41f3a4498ea402b9f639" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52827539fa5a41f3a4498ea402b9f639" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 35185 } } }
I20260928 14:41:54.671723 9301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:54.672076 9308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:41:54.672623 9308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:41:54.675035 9308 catalog_manager.cc:1347] Generated new cluster ID: 0c0705a54d1348169aa5117b8afa9eeb
I20260928 14:41:54.675109 9308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:41:54 dist-test-slave-wwwb krb5kdc[9215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:41:54.680948 9308 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:41:54.681700 9308 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:41:54.684036 9304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:54.671302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54650 (local address 127.22.13.126:35185)
0928 14:41:54.672110 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:54.672121 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:54.672167 (+ 46us) server_negotiation.cc:407] Connection header received
0928 14:41:54.672306 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:54.672315 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:54.672381 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:54.672568 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:54.673731 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:54.674878 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:54.675970 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:54.676644 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:54.676769 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:54.679786 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:54.679791 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:54.679791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:54.679805 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:54.682171 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:54.682630 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:54.682633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:54.682634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:54.682747 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:54.682956 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:54.682957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:54.682957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:54.683244 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:54.683400 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:54.683511 (+ 111us) server_negotiation.cc:299] Negotiation successful
0928 14:41:54.683776 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751,"thread_start_us":80,"threads_started":1}
I20260928 14:41:54.690238 9308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639: Generated new TSK 0
I20260928 14:41:54.690480 9308 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.13.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
Entry for principal kudu/127.22.13.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab.
I20260928 14:41:54.720534 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35185
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:39213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:54.804981 9322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:54.805272 9322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:54.805322 9322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:54.806686 9322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:54.806738 9322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:54.806787 9322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:41:54.808370 9322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:39213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.65.keytab
--principal=kudu/127.22.13.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:54.808604 9322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:54.808833 9322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:41:54.814244 9322 server_base.cc:1034] running on GCE node
W20260928 14:41:54.814208 9327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:54.814225 9328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:54.814217 9330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:54.814616 9322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:54.814922 9322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:54.816082 9322 hybrid_clock.cc:648] HybridClock initialized: now 1790606514816073 us; error 37 us; skew 500 ppm
Sep 28 14:41:54 dist-test-slave-wwwb krb5kdc[9215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606514, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:54.818665 9322 init.cc:377] Logged in from keytab as kudu/127.22.13.65@KRBTEST.COM (short username kudu)
I20260928 14:41:54.819187 9322 webserver.cc:460] Webserver started at http://127.22.13.65:41085/ using document root <none> and password file <none>
I20260928 14:41:54.819448 9322 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:54.819521 9322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:54.819676 9322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:54.820775 9322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2"
format_stamp: "Formatted at 2026-09-28 14:41:54 on dist-test-slave-wwwb"
server_key: "466eb70b45f201fffb75faa1ef85194f"
server_key_iv: "a8760829fc4adcb26cfde36a9f1b10e3"
server_key_version: "encryptionkey@0"
I20260928 14:41:54.821174 9322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2"
format_stamp: "Formatted at 2026-09-28 14:41:54 on dist-test-slave-wwwb"
server_key: "466eb70b45f201fffb75faa1ef85194f"
server_key_iv: "a8760829fc4adcb26cfde36a9f1b10e3"
server_key_version: "encryptionkey@0"
I20260928 14:41:54.822641 9322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:41:54.823450 9337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:54.823624 9322 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:54.823721 9322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2"
format_stamp: "Formatted at 2026-09-28 14:41:54 on dist-test-slave-wwwb"
server_key: "466eb70b45f201fffb75faa1ef85194f"
server_key_iv: "a8760829fc4adcb26cfde36a9f1b10e3"
server_key_version: "encryptionkey@0"
I20260928 14:41:54.823805 9322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:54.855978 9322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:54.858016 9322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:54.858227 9322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:54.858485 9322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:54.858809 9322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:54.858865 9322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:54.858901 9322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:54.858949 9322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:54.864863 9322 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:39009
I20260928 14:41:54.864915 9450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:39009 every 8 connection(s)
I20260928 14:41:54.865255 9322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:41:54.866175 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 9322
I20260928 14:41:54.866297 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:41:54.866528 22581 external_mini_cluster.cc:1428] Setting key 6c449d216fd82bd5d15fd08bc5af3365
Sep 28 14:41:54 dist-test-slave-wwwb krb5kdc[9215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606514, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.65@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.13.66@KRBTEST.COM; defaulting to no policy
I20260928 14:41:54.876827 9304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:54.866048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:55235 (local address 127.22.13.126:35185)
0928 14:41:54.866105 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:54.866107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:54.866383 (+ 276us) server_negotiation.cc:407] Connection header received
0928 14:41:54.867046 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:54.867047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:54.867059 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:54.867108 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:54.868511 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:54.869000 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:54.870584 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:54.870944 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:54.871036 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:54.873684 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:54.873686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:54.873687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:54.873697 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:54.875390 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:54.875813 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:54.875815 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:54.875816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:54.875880 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:54.876136 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:54.876137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:54.876137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:54.876282 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:54.876353 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:54.876621 (+ 268us) server_negotiation.cc:299] Negotiation successful
0928 14:41:54.876729 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:41:54.877014 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:54.866061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:35185 (local address 127.22.13.65:55235)
0928 14:41:54.866228 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:54.866289 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:54.866921 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:54.867193 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:54.867196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:54.868104 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:54.868400 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:54.868407 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:54.869102 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:54.869103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:54.870483 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:54.870486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:54.871038 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:54.871038 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:54.871145 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:54.871834 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:41:54.871846 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:54.873573 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:54.875495 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:54.875503 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:54.875505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:54.875733 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:54.875962 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:54.875963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:54.875963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:54.876066 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:54.876347 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:54.876351 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:54.876479 (+ 128us) client_negotiation.cc:769] Sending connection context
0928 14:41:54.876594 (+ 115us) client_negotiation.cc:240] Negotiation successful
0928 14:41:54.876761 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
I20260928 14:41:54.877519 9451 heartbeater.cc:347] Connected to a master server at 127.22.13.126:35185
I20260928 14:41:54.877614 9451 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:54.877769 9451 heartbeater.cc:511] Master 127.22.13.126:35185 requested a full tablet report, sending...
I20260928 14:41:54.878356 9260 ts_manager.cc:194] Registered new tserver with Master: f5ee72ecf0174fb3bbf5ca26a2e4a8b2 (127.22.13.65:39009)
I20260928 14:41:54.879043 9260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.65@KRBTEST.COM'} at 127.22.13.65:55235
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
Entry for principal kudu/127.22.13.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab.
I20260928 14:41:54.894323 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35185
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:39213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:54.978389 9456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:54.978650 9456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:54.978695 9456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:54.979909 9456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:54.979954 9456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:54.979982 9456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:41:54.981397 9456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:39213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.66.keytab
--principal=kudu/127.22.13.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:54.981606 9456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:54.981802 9456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:54.986994 9461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:54.986994 9464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:54.986996 9462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:54.987391 9456 server_base.cc:1034] running on GCE node
I20260928 14:41:54.987646 9456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:54.987967 9456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:54.989128 9456 hybrid_clock.cc:648] HybridClock initialized: now 1790606514989100 us; error 41 us; skew 500 ppm
Sep 28 14:41:54 dist-test-slave-wwwb krb5kdc[9215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606514, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:54.991717 9456 init.cc:377] Logged in from keytab as kudu/127.22.13.66@KRBTEST.COM (short username kudu)
I20260928 14:41:54.992226 9456 webserver.cc:460] Webserver started at http://127.22.13.66:38387/ using document root <none> and password file <none>
I20260928 14:41:54.992530 9456 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:54.992612 9456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:54.992753 9456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:54.993861 9456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "3d32848c87944c8a8cce33737713e7f4"
format_stamp: "Formatted at 2026-09-28 14:41:54 on dist-test-slave-wwwb"
server_key: "8b543d5847789b9cada9166013f3b65a"
server_key_iv: "882df9bb79857e63d7449c4056101d58"
server_key_version: "encryptionkey@0"
I20260928 14:41:54.994304 9456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "3d32848c87944c8a8cce33737713e7f4"
format_stamp: "Formatted at 2026-09-28 14:41:54 on dist-test-slave-wwwb"
server_key: "8b543d5847789b9cada9166013f3b65a"
server_key_iv: "882df9bb79857e63d7449c4056101d58"
server_key_version: "encryptionkey@0"
I20260928 14:41:54.995694 9456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:54.996455 9471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:54.996635 9456 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:54.996738 9456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "3d32848c87944c8a8cce33737713e7f4"
format_stamp: "Formatted at 2026-09-28 14:41:54 on dist-test-slave-wwwb"
server_key: "8b543d5847789b9cada9166013f3b65a"
server_key_iv: "882df9bb79857e63d7449c4056101d58"
server_key_version: "encryptionkey@0"
I20260928 14:41:54.996829 9456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:55.007010 9456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:55.008944 9456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:55.009137 9456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:55.009389 9456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:55.009727 9456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:55.009783 9456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:55.009824 9456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:55.009855 9456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:55.016172 9456 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:36383
I20260928 14:41:55.016227 9584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:36383 every 8 connection(s)
I20260928 14:41:55.016572 9456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:41:55.019443 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 9456
I20260928 14:41:55.019572 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:41:55.019788 22581 external_mini_cluster.cc:1428] Setting key a17e17726d52b1b687833c4a39d99c70
Sep 28 14:41:55 dist-test-slave-wwwb krb5kdc[9215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606514, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.66@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:41:55.027350 9304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.017334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:33615 (local address 127.22.13.126:35185)
0928 14:41:55.017404 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.017406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.017651 (+ 245us) server_negotiation.cc:407] Connection header received
0928 14:41:55.018235 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.018236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.018246 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.018320 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:55.019301 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.020070 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.021163 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.021484 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.021561 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.024273 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:55.024275 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:55.024275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:55.024284 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:55.025753 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:55.026269 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:55.026270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:55.026270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:55.026334 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:55.026605 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:55.026606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:55.026606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:55.026799 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:55.026868 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.027169 (+ 301us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.027258 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:41:55.027555 9587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.017327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:35185 (local address 127.22.13.66:33615)
0928 14:41:55.017521 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:55.017560 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:55.018143 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:55.018396 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:55.018400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:55.018987 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:55.019209 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.019214 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.020171 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.020172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.021075 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.021078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.021575 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.021576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.021681 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.022275 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:41:55.022287 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:55.024161 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:55.025847 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:55.025855 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:55.025857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:55.026175 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:55.026406 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:55.026407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:55.026407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:55.026530 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:55.026870 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:55.026878 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:55.027034 (+ 156us) client_negotiation.cc:769] Sending connection context
0928 14:41:55.027138 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 14:41:55.027307 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":114,"threads_started":1}
I20260928 14:41:55.028040 9585 heartbeater.cc:347] Connected to a master server at 127.22.13.126:35185
I20260928 14:41:55.028141 9585 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:55.028337 9585 heartbeater.cc:511] Master 127.22.13.126:35185 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.13.67@KRBTEST.COM; defaulting to no policy
I20260928 14:41:55.028777 9260 ts_manager.cc:194] Registered new tserver with Master: 3d32848c87944c8a8cce33737713e7f4 (127.22.13.66:36383)
I20260928 14:41:55.029167 9260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.66@KRBTEST.COM'} at 127.22.13.66:33615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
Entry for principal kudu/127.22.13.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab.
I20260928 14:41:55.044616 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35185
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:39213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:55.128351 9590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:55.128572 9590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:55.128618 9590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:55.129899 9590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:55.130000 9590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:55.130048 9590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:41:55.131537 9590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:39213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.67.keytab
--principal=kudu/127.22.13.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:35185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:55.131742 9590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:55.131943 9590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:41:55.137532 9590 server_base.cc:1034] running on GCE node
W20260928 14:41:55.137511 9596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:55.137482 9595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:55.137512 9598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:55.138008 9590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:55.138280 9590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:55.139430 9590 hybrid_clock.cc:648] HybridClock initialized: now 1790606515139416 us; error 32 us; skew 500 ppm
Sep 28 14:41:55 dist-test-slave-wwwb krb5kdc[9215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606515, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:55.142227 9590 init.cc:377] Logged in from keytab as kudu/127.22.13.67@KRBTEST.COM (short username kudu)
I20260928 14:41:55.142707 9590 webserver.cc:460] Webserver started at http://127.22.13.67:43567/ using document root <none> and password file <none>
I20260928 14:41:55.142913 9590 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:55.142962 9590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:55.143054 9590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:55.144153 9590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "17bc68fc2fe34646ba86e6691308db23"
format_stamp: "Formatted at 2026-09-28 14:41:55 on dist-test-slave-wwwb"
server_key: "9c2b03221d92129acfb22c8cbc4c11b9"
server_key_iv: "b60d270237e3e525194dbd33ad541db8"
server_key_version: "encryptionkey@0"
I20260928 14:41:55.144459 9590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "17bc68fc2fe34646ba86e6691308db23"
format_stamp: "Formatted at 2026-09-28 14:41:55 on dist-test-slave-wwwb"
server_key: "9c2b03221d92129acfb22c8cbc4c11b9"
server_key_iv: "b60d270237e3e525194dbd33ad541db8"
server_key_version: "encryptionkey@0"
I20260928 14:41:55.145797 9590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:41:55.146719 9605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:55.146898 9590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:41:55.147014 9590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "17bc68fc2fe34646ba86e6691308db23"
format_stamp: "Formatted at 2026-09-28 14:41:55 on dist-test-slave-wwwb"
server_key: "9c2b03221d92129acfb22c8cbc4c11b9"
server_key_iv: "b60d270237e3e525194dbd33ad541db8"
server_key_version: "encryptionkey@0"
I20260928 14:41:55.147094 9590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:55.156638 9590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:55.159149 9590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:55.159345 9590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:55.159593 9590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:55.159911 9590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:55.159971 9590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:55.160015 9590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:55.160060 9590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:55.166149 9590 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:33019
I20260928 14:41:55.166190 9718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:33019 every 8 connection(s)
I20260928 14:41:55.166553 9590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:41:55.169755 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 9590
I20260928 14:41:55.169863 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:41:55.170133 22581 external_mini_cluster.cc:1428] Setting key b601290837b838b0e59806a696663b93
Sep 28 14:41:55 dist-test-slave-wwwb krb5kdc[9215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606515, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.67@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:41:55.176769 9304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.167472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:39219 (local address 127.22.13.126:35185)
0928 14:41:55.167551 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.167554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.167794 (+ 240us) server_negotiation.cc:407] Connection header received
0928 14:41:55.168286 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.168288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.168298 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.168354 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:55.169365 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.169919 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.171568 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.171763 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.171845 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.174126 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:55.174128 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:55.174128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:55.174136 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:55.175583 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:55.175953 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:55.175954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:55.175954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:55.176019 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:55.176230 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:55.176231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:55.176231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:55.176336 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:55.176384 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.176636 (+ 252us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.176701 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 14:41:55.176994 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.167478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:35185 (local address 127.22.13.67:39219)
0928 14:41:55.167665 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:55.167705 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:55.168199 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:55.168425 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:55.168429 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:55.169043 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:55.169272 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.169276 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.170110 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.170112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.171481 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.171485 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.171843 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.171844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.171957 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.172540 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:41:55.172549 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:55.174017 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:55.175660 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:55.175666 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:55.175668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:55.175887 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:55.176077 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:41:55.176078 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:41:55.176078 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:41:55.176177 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:41:55.176389 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:55.176393 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:55.176523 (+ 130us) client_negotiation.cc:769] Sending connection context
0928 14:41:55.176607 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 14:41:55.176735 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":86,"threads_started":1}
I20260928 14:41:55.177572 9719 heartbeater.cc:347] Connected to a master server at 127.22.13.126:35185
I20260928 14:41:55.177662 9719 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:55.177809 9719 heartbeater.cc:511] Master 127.22.13.126:35185 requested a full tablet report, sending...
I20260928 14:41:55.178177 9260 ts_manager.cc:194] Registered new tserver with Master: 17bc68fc2fe34646ba86e6691308db23 (127.22.13.67:33019)
I20260928 14:41:55.178560 9260 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.13.67@KRBTEST.COM'} at 127.22.13.67:39219
I20260928 14:41:55.181869 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:41:55.189029 9304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.183330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54662 (local address 127.22.13.126:35185)
0928 14:41:55.183394 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.183396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.183469 (+ 73us) server_negotiation.cc:407] Connection header received
0928 14:41:55.183534 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.183534 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.183543 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.183587 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:55.184245 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.184700 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.185371 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.185541 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.185598 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.186747 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:55.186748 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:55.186748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:55.186755 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:55.188109 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:55.188401 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:55.188402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:55.188402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:55.188452 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:55.188596 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:55.188597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:55.188597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:55.188734 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:55.188803 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.188892 (+ 89us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.188954 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:41:55.190974 9260 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:41:55.192600 9260 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:41:55.199515 9731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.195214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:33019 (local address 127.0.0.1:57664)
0928 14:41:55.195408 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:55.195447 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:55.195516 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:55.196034 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:55.196037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:55.196040 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:55.196140 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.196142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.197061 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.197062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.198238 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.198241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.199172 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.199173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.199257 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.199272 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 14:41:55.199348 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 14:41:55.199421 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":55,"thread_start_us":84,"threads_started":1}
I20260928 14:41:55.199532 9733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.195682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57664 (local address 127.22.13.67:33019)
0928 14:41:55.195851 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.195856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.195866 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:41:55.195904 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.195906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.195921 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.196221 (+ 300us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:55.196298 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.196937 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.198326 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.199075 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.199147 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.199210 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.199392 (+ 182us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.199448 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260928 14:41:55.200367 9653 tablet_service.cc:1467] Processing CreateTablet for tablet 9576583bf6e74e87a5057bcf5b242aba (DEFAULT_TABLE table=test-table [id=f77dc2f5e6bd4388b0119842ac7007cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:55.200776 9653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9576583bf6e74e87a5057bcf5b242aba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:55.201109 9730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.195447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:39009 (local address 127.0.0.1:38602)
0928 14:41:55.196193 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:55.196204 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:55.196250 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:55.197363 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:55.197364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:55.197366 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:55.197411 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.197413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.198671 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.198672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.199837 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.199840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.200752 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.200752 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.200967 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.200971 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:41:55.201014 (+ 43us) client_negotiation.cc:240] Negotiation successful
0928 14:41:55.201058 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
I20260928 14:41:55.201211 9735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.196035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38602 (local address 127.22.13.65:39009)
0928 14:41:55.197138 (+ 1103us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.197142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.197152 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:41:55.197189 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.197191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.197205 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.197278 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:55.197519 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.198576 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.199924 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.200661 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.200738 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.200882 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.201066 (+ 184us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.201108 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1063,"thread_start_us":72,"threads_started":1}
I20260928 14:41:55.202023 9734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.195764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49004 (local address 127.22.13.66:36383)
0928 14:41:55.195976 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.195981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.195992 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:41:55.196033 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.196035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.196051 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.196140 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:55.196237 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.196941 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.198463 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.199146 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.199205 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.199253 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.199495 (+ 242us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.199559 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":75,"threads_started":1}
I20260928 14:41:55.201879 9385 tablet_service.cc:1467] Processing CreateTablet for tablet 9576583bf6e74e87a5057bcf5b242aba (DEFAULT_TABLE table=test-table [id=f77dc2f5e6bd4388b0119842ac7007cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:55.202255 9732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.195240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:36383 (local address 127.0.0.1:49004)
0928 14:41:55.195602 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:55.195611 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:55.195655 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:55.196146 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:55.196147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:55.196148 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:55.196184 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.196186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.197046 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.197047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.198362 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.198365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.199293 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.199294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.199398 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.199401 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:55.202124 (+ 2723us) client_negotiation.cc:240] Negotiation successful
0928 14:41:55.202189 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":34,"thread_start_us":79,"threads_started":1}
I20260928 14:41:55.202279 9385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9576583bf6e74e87a5057bcf5b242aba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:55.204381 9736 tablet_bootstrap.cc:492] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23: Bootstrap starting.
I20260928 14:41:55.205014 9736 tablet_bootstrap.cc:654] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:55.205430 9737 tablet_bootstrap.cc:492] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2: Bootstrap starting.
I20260928 14:41:55.205610 9736 log.cc:824] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:55.206154 9737 tablet_bootstrap.cc:654] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:55.206154 9519 tablet_service.cc:1467] Processing CreateTablet for tablet 9576583bf6e74e87a5057bcf5b242aba (DEFAULT_TABLE table=test-table [id=f77dc2f5e6bd4388b0119842ac7007cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:55.206573 9519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9576583bf6e74e87a5057bcf5b242aba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:55.206732 9736 tablet_bootstrap.cc:492] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23: No bootstrap required, opened a new log
I20260928 14:41:55.206724 9737 log.cc:824] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:55.206821 9736 ts_tablet_manager.cc:1397] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:41:55.207520 9737 tablet_bootstrap.cc:492] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2: No bootstrap required, opened a new log
I20260928 14:41:55.207593 9737 ts_tablet_manager.cc:1397] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:41:55.208469 9736 raft_consensus.cc:348] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17bc68fc2fe34646ba86e6691308db23" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33019 } } peers { permanent_uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 39009 } } peers { permanent_uuid: "3d32848c87944c8a8cce33737713e7f4" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36383 } }
I20260928 14:41:55.208644 9736 raft_consensus.cc:374] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:55.208703 9736 raft_consensus.cc:729] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17bc68fc2fe34646ba86e6691308db23, State: Initialized, Role: FOLLOWER
I20260928 14:41:55.208860 9736 consensus_queue.cc:260] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17bc68fc2fe34646ba86e6691308db23" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33019 } } peers { permanent_uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 39009 } } peers { permanent_uuid: "3d32848c87944c8a8cce33737713e7f4" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36383 } }
I20260928 14:41:55.209177 9736 ts_tablet_manager.cc:1428] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:41:55.209196 9719 heartbeater.cc:503] Master 127.22.13.126:35185 was elected leader, sending a full tablet report...
I20260928 14:41:55.209740 9737 raft_consensus.cc:348] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17bc68fc2fe34646ba86e6691308db23" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33019 } } peers { permanent_uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 39009 } } peers { permanent_uuid: "3d32848c87944c8a8cce33737713e7f4" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36383 } }
I20260928 14:41:55.209929 9737 raft_consensus.cc:374] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:55.210013 9737 raft_consensus.cc:729] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ee72ecf0174fb3bbf5ca26a2e4a8b2, State: Initialized, Role: FOLLOWER
I20260928 14:41:55.210139 9737 consensus_queue.cc:260] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17bc68fc2fe34646ba86e6691308db23" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33019 } } peers { permanent_uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 39009 } } peers { permanent_uuid: "3d32848c87944c8a8cce33737713e7f4" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36383 } }
I20260928 14:41:55.210426 9737 ts_tablet_manager.cc:1428] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:41:55.210588 9451 heartbeater.cc:503] Master 127.22.13.126:35185 was elected leader, sending a full tablet report...
I20260928 14:41:55.210906 9742 tablet_bootstrap.cc:492] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4: Bootstrap starting.
I20260928 14:41:55.211606 9742 tablet_bootstrap.cc:654] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:55.212085 9742 log.cc:824] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:55.212837 9742 tablet_bootstrap.cc:492] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4: No bootstrap required, opened a new log
I20260928 14:41:55.212975 9742 ts_tablet_manager.cc:1397] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:41:55.214517 9742 raft_consensus.cc:348] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17bc68fc2fe34646ba86e6691308db23" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33019 } } peers { permanent_uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 39009 } } peers { permanent_uuid: "3d32848c87944c8a8cce33737713e7f4" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36383 } }
I20260928 14:41:55.214681 9742 raft_consensus.cc:374] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:55.214728 9742 raft_consensus.cc:729] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d32848c87944c8a8cce33737713e7f4, State: Initialized, Role: FOLLOWER
I20260928 14:41:55.214874 9742 consensus_queue.cc:260] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17bc68fc2fe34646ba86e6691308db23" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33019 } } peers { permanent_uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 39009 } } peers { permanent_uuid: "3d32848c87944c8a8cce33737713e7f4" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36383 } }
I20260928 14:41:55.215143 9742 ts_tablet_manager.cc:1428] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:41:55.215188 9585 heartbeater.cc:503] Master 127.22.13.126:35185 was elected leader, sending a full tablet report...
I20260928 14:41:55.257797 9741 raft_consensus.cc:482] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:41:55.258029 9741 raft_consensus.cc:504] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17bc68fc2fe34646ba86e6691308db23" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33019 } } peers { permanent_uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 39009 } } peers { permanent_uuid: "3d32848c87944c8a8cce33737713e7f4" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36383 } }
I20260928 14:41:55.258380 9741 leader_election.cc:288] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17bc68fc2fe34646ba86e6691308db23 (127.22.13.67:33019), 3d32848c87944c8a8cce33737713e7f4 (127.22.13.66:36383)
I20260928 14:41:55.262313 9734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.258672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:41751 (local address 127.22.13.66:36383)
0928 14:41:55.258729 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.258731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.258742 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:41:55.258772 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.258773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.258788 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.258847 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:55.259072 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.259861 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.261353 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.261917 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.262016 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.262025 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.262196 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.262240 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260928 14:41:55.262313 9733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.258679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:48997 (local address 127.22.13.67:33019)
0928 14:41:55.258732 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.258734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.259046 (+ 312us) server_negotiation.cc:407] Connection header received
0928 14:41:55.259091 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.259093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.259103 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.259159 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:55.259287 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.259961 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.261352 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.261917 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.262015 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.262026 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.262196 (+ 170us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.262239 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:41:55.262315 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.258586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:36383 (local address 127.22.13.65:41751)
0928 14:41:55.258633 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:55.258646 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:55.258692 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:55.258889 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:55.258891 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:55.258894 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:55.258979 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.258981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.259965 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.259966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.261242 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.261246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.262032 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.262033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.262117 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.262119 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:41:55.262173 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 14:41:55.262224 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":13}
I20260928 14:41:55.262320 9745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.258586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:33019 (local address 127.22.13.65:48997)
0928 14:41:55.258939 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:55.258951 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:55.259037 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:55.259169 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:55.259170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:55.259171 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:55.259221 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.259222 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.260038 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.260039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.261242 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.261246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.262045 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.262046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.262114 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.262118 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:41:55.262173 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 14:41:55.262224 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":197,"thread_start_us":123,"threads_started":1}
I20260928 14:41:55.262748 9673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9576583bf6e74e87a5057bcf5b242aba" candidate_uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17bc68fc2fe34646ba86e6691308db23" is_pre_election: true
I20260928 14:41:55.262712 9539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9576583bf6e74e87a5057bcf5b242aba" candidate_uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d32848c87944c8a8cce33737713e7f4" is_pre_election: true
I20260928 14:41:55.262904 9539 raft_consensus.cc:2454] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ee72ecf0174fb3bbf5ca26a2e4a8b2 in term 0.
I20260928 14:41:55.262912 9673 raft_consensus.cc:2454] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ee72ecf0174fb3bbf5ca26a2e4a8b2 in term 0.
I20260928 14:41:55.263186 9338 leader_election.cc:302] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17bc68fc2fe34646ba86e6691308db23, f5ee72ecf0174fb3bbf5ca26a2e4a8b2; no voters:
I20260928 14:41:55.263370 9741 raft_consensus.cc:2781] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:41:55.263422 9741 raft_consensus.cc:482] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:41:55.263437 9741 raft_consensus.cc:3037] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:55.264142 9741 raft_consensus.cc:504] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17bc68fc2fe34646ba86e6691308db23" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33019 } } peers { permanent_uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 39009 } } peers { permanent_uuid: "3d32848c87944c8a8cce33737713e7f4" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36383 } }
I20260928 14:41:55.264300 9741 leader_election.cc:288] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [CANDIDATE]: Term 1 election: Requested vote from peers 17bc68fc2fe34646ba86e6691308db23 (127.22.13.67:33019), 3d32848c87944c8a8cce33737713e7f4 (127.22.13.66:36383)
I20260928 14:41:55.264456 9673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9576583bf6e74e87a5057bcf5b242aba" candidate_uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17bc68fc2fe34646ba86e6691308db23"
I20260928 14:41:55.264542 9673 raft_consensus.cc:3037] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:55.264510 9539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9576583bf6e74e87a5057bcf5b242aba" candidate_uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d32848c87944c8a8cce33737713e7f4"
I20260928 14:41:55.264612 9539 raft_consensus.cc:3037] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:55.265308 9673 raft_consensus.cc:2454] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ee72ecf0174fb3bbf5ca26a2e4a8b2 in term 1.
I20260928 14:41:55.265429 9539 raft_consensus.cc:2454] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ee72ecf0174fb3bbf5ca26a2e4a8b2 in term 1.
I20260928 14:41:55.265491 9338 leader_election.cc:302] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17bc68fc2fe34646ba86e6691308db23, f5ee72ecf0174fb3bbf5ca26a2e4a8b2; no voters:
I20260928 14:41:55.265594 9741 raft_consensus.cc:2781] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:41:55.265797 9741 raft_consensus.cc:686] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [term 1 LEADER]: Becoming Leader. State: Replica: f5ee72ecf0174fb3bbf5ca26a2e4a8b2, State: Running, Role: LEADER
I20260928 14:41:55.265911 9741 consensus_queue.cc:237] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17bc68fc2fe34646ba86e6691308db23" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33019 } } peers { permanent_uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 39009 } } peers { permanent_uuid: "3d32848c87944c8a8cce33737713e7f4" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36383 } }
I20260928 14:41:55.266836 9259 catalog_manager.cc:5495] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5ee72ecf0174fb3bbf5ca26a2e4a8b2 (127.22.13.65). New cstate: current_term: 1 leader_uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17bc68fc2fe34646ba86e6691308db23" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5ee72ecf0174fb3bbf5ca26a2e4a8b2" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 39009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d32848c87944c8a8cce33737713e7f4" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36383 } health_report { overall_health: UNKNOWN } } }
W20260928 14:41:55.267120 9586 tablet.cc:2369] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:41:55.272329 9735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.269879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38608 (local address 127.22.13.65:39009)
0928 14:41:55.269923 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.269925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.269969 (+ 44us) server_negotiation.cc:407] Connection header received
0928 14:41:55.270029 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.270030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.270066 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.270124 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:41:55.270219 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.270754 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.271609 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.271802 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.271871 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.271992 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:41:55.272089 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:41:55.272145 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.272179 (+ 34us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.272197 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:41:55.274338 9539 raft_consensus.cc:1261] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4 [term 1 FOLLOWER]: Refusing update from remote peer f5ee72ecf0174fb3bbf5ca26a2e4a8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:41:55.274662 9673 raft_consensus.cc:1261] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23 [term 1 FOLLOWER]: Refusing update from remote peer f5ee72ecf0174fb3bbf5ca26a2e4a8b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:41:55.274734 9746 consensus_queue.cc:1035] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d32848c87944c8a8cce33737713e7f4" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 36383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:41:55.274935 9746 consensus_queue.cc:1035] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17bc68fc2fe34646ba86e6691308db23" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:41:55.277180 9749 mvcc.cc:205] Tried to move back new op lower bound from 7334324286560477184 to 7334324286529941504. Current Snapshot: MvccSnapshot[applied={T|T < 7334324286560477184}]
I20260928 14:41:55.277876 9750 mvcc.cc:205] Tried to move back new op lower bound from 7334324286560477184 to 7334324286529941504. Current Snapshot: MvccSnapshot[applied={T|T < 7334324286560477184 or (T in {7334324286560477184})}]
I20260928 14:41:55.279731 9260 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:41:55.279820 9260 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:41:55.280650 9260 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52827539fa5a41f3a4498ea402b9f639: Sending DeleteTablet for 3 replicas of tablet 9576583bf6e74e87a5057bcf5b242aba
I20260928 14:41:55.281004 9385 tablet_service.cc:1514] Processing DeleteTablet for tablet 9576583bf6e74e87a5057bcf5b242aba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:55 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:38602
I20260928 14:41:55.281044 9653 tablet_service.cc:1514] Processing DeleteTablet for tablet 9576583bf6e74e87a5057bcf5b242aba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:55 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:57664
I20260928 14:41:55.281247 9759 tablet_replica.cc:330] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2: stopping tablet replica
I20260928 14:41:55.281308 9519 tablet_service.cc:1514] Processing DeleteTablet for tablet 9576583bf6e74e87a5057bcf5b242aba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:55 UTC) from {username='kudu', principal='kudu/127.22.13.126@KRBTEST.COM'} at 127.0.0.1:49004
I20260928 14:41:55.281345 9759 raft_consensus.cc:2229] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:41:55.281448 9759 raft_consensus.cc:2258] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:55.281639 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 9322
I20260928 14:41:55.281509 9761 tablet_replica.cc:330] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4: stopping tablet replica
I20260928 14:41:55.281881 9759 ts_tablet_manager.cc:1905] T 9576583bf6e74e87a5057bcf5b242aba P f5ee72ecf0174fb3bbf5ca26a2e4a8b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:55.282356 9761 raft_consensus.cc:2229] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:41:55.282451 9761 raft_consensus.cc:2258] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:55.282842 9761 ts_tablet_manager.cc:1905] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:55.283768 9760 tablet_replica.cc:330] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23: stopping tablet replica
I20260928 14:41:55.283893 9760 raft_consensus.cc:2229] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:41:55.284020 9760 raft_consensus.cc:2258] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:55.284399 9761 ts_tablet_manager.cc:1918] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:41:55.284471 9761 log.cc:1196] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-1/wal/wals/9576583bf6e74e87a5057bcf5b242aba
I20260928 14:41:55.284523 9760 ts_tablet_manager.cc:1905] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:55.284757 9761 ts_tablet_manager.cc:1939] T 9576583bf6e74e87a5057bcf5b242aba P 3d32848c87944c8a8cce33737713e7f4: Deleting consensus metadata
I20260928 14:41:55.285171 9246 catalog_manager.cc:4841] TS 3d32848c87944c8a8cce33737713e7f4 (127.22.13.66:36383): tablet 9576583bf6e74e87a5057bcf5b242aba (table test-table [id=f77dc2f5e6bd4388b0119842ac7007cd]) successfully deleted
I20260928 14:41:55.286489 9760 ts_tablet_manager.cc:1918] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:41:55.286569 9760 log.cc:1196] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/9576583bf6e74e87a5057bcf5b242aba
I20260928 14:41:55.286836 9760 ts_tablet_manager.cc:1939] T 9576583bf6e74e87a5057bcf5b242aba P 17bc68fc2fe34646ba86e6691308db23: Deleting consensus metadata
I20260928 14:41:55.287254 9244 catalog_manager.cc:4841] TS 17bc68fc2fe34646ba86e6691308db23 (127.22.13.67:33019): tablet 9576583bf6e74e87a5057bcf5b242aba (table test-table [id=f77dc2f5e6bd4388b0119842ac7007cd]) successfully deleted
W20260928 14:41:55.288231 9244 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:39009 (error 108)
I20260928 14:41:55.288482 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 9456
I20260928 14:41:55.288806 9732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.288684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:39009 (local address 127.0.0.1:38614)
0928 14:41:55.288732 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:55.288766 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:39009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260928 14:41:55.289438 9244 catalog_manager.cc:4584] TS f5ee72ecf0174fb3bbf5ca26a2e4a8b2 (127.22.13.65:39009): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9576583bf6e74e87a5057bcf5b242aba: Network error: Client connection negotiation failed: client connection to 127.22.13.65:39009: connect: Connection refused (error 111)
I20260928 14:41:55.293879 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 9590
I20260928 14:41:55.298565 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 9228
2026-09-28T14:41:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1916 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-28T14:41:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:41:55Z Disabled control of system clock
I20260928 14:41:55.327634 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:46747
--webserver_interface=127.22.13.126
--webserver_port=0
--builtin_ntp_servers=127.22.13.84:44455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:46747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:41:55.410744 9766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:55.411017 9766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:55.411041 9766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:55.412407 9766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:41:55.412459 9766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:55.412472 9766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:55.412487 9766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:41:55.412498 9766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:41:55.414080 9766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:46747
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:46747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:55.414315 9766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:55.414587 9766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:55.419711 9771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:55.419711 9774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:55.419711 9772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:55.419987 9766 server_base.cc:1034] running on GCE node
I20260928 14:41:55.420223 9766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:55.420516 9766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:55.421670 9766 hybrid_clock.cc:648] HybridClock initialized: now 1790606515421653 us; error 33 us; skew 500 ppm
I20260928 14:41:55.423077 9766 webserver.cc:460] Webserver started at http://127.22.13.126:36811/ using document root <none> and password file <none>
I20260928 14:41:55.423341 9766 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:55.423389 9766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:55.423549 9766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:55.424765 9766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "98f604f4c4f34fdab56d588b3dd8cc2e"
format_stamp: "Formatted at 2026-09-28 14:41:55 on dist-test-slave-wwwb"
server_key: "487540d48ec660155677c5126f1ae737"
server_key_iv: "8de5b38eabbd4ccabd1940c9cd6db95e"
server_key_version: "encryptionkey@0"
I20260928 14:41:55.425153 9766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "98f604f4c4f34fdab56d588b3dd8cc2e"
format_stamp: "Formatted at 2026-09-28 14:41:55 on dist-test-slave-wwwb"
server_key: "487540d48ec660155677c5126f1ae737"
server_key_iv: "8de5b38eabbd4ccabd1940c9cd6db95e"
server_key_version: "encryptionkey@0"
I20260928 14:41:55.426632 9766 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:41:55.427388 9780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:55.427575 9766 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:55.427691 9766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "98f604f4c4f34fdab56d588b3dd8cc2e"
format_stamp: "Formatted at 2026-09-28 14:41:55 on dist-test-slave-wwwb"
server_key: "487540d48ec660155677c5126f1ae737"
server_key_iv: "8de5b38eabbd4ccabd1940c9cd6db95e"
server_key_version: "encryptionkey@0"
I20260928 14:41:55.427774 9766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:55.445529 9766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:55.445919 9766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:55.446172 9766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:55.450416 9766 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:46747
I20260928 14:41:55.450454 9832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:46747 every 8 connection(s)
I20260928 14:41:55.450838 9766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:41:55.451817 9833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:55.452476 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 9766
I20260928 14:41:55.452602 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:41:55.452800 22581 external_mini_cluster.cc:1428] Setting key 625f6afea4ec4a3f7c5def384530cd1d
I20260928 14:41:55.455001 9833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e: Bootstrap starting.
I20260928 14:41:55.455672 9833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:55.456013 9833 log.cc:824] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:55.456789 9833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e: No bootstrap required, opened a new log
I20260928 14:41:55.458705 9836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.453408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48102 (local address 127.22.13.126:46747)
0928 14:41:55.453601 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.453609 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.453635 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:41:55.454124 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.454132 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.454198 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.454320 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:55.454570 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.455484 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.456564 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.457041 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.457132 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.457827 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:55.457832 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:55.457832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:55.458080 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:55.458115 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:55.458127 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:55.458178 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.458280 (+ 102us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.458348 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":93,"threads_started":1}
I20260928 14:41:55.458760 9833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f604f4c4f34fdab56d588b3dd8cc2e" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46747 } }
I20260928 14:41:55.458962 9833 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:55.459005 9833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98f604f4c4f34fdab56d588b3dd8cc2e, State: Initialized, Role: FOLLOWER
I20260928 14:41:55.459146 9833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f604f4c4f34fdab56d588b3dd8cc2e" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46747 } }
I20260928 14:41:55.459226 9833 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:41:55.459256 9833 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:41:55.459301 9833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:55.459887 9833 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f604f4c4f34fdab56d588b3dd8cc2e" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46747 } }
I20260928 14:41:55.459978 9833 leader_election.cc:302] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98f604f4c4f34fdab56d588b3dd8cc2e; no voters:
I20260928 14:41:55.460124 9833 leader_election.cc:288] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:41:55.460206 9838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:41:55.460355 9838 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [term 1 LEADER]: Becoming Leader. State: Replica: 98f604f4c4f34fdab56d588b3dd8cc2e, State: Running, Role: LEADER
I20260928 14:41:55.460412 9833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:41:55.460453 9838 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f604f4c4f34fdab56d588b3dd8cc2e" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46747 } }
I20260928 14:41:55.460879 9840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98f604f4c4f34fdab56d588b3dd8cc2e. Latest consensus state: current_term: 1 leader_uuid: "98f604f4c4f34fdab56d588b3dd8cc2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f604f4c4f34fdab56d588b3dd8cc2e" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46747 } } }
I20260928 14:41:55.460994 9840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:55.460866 9839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98f604f4c4f34fdab56d588b3dd8cc2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f604f4c4f34fdab56d588b3dd8cc2e" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 46747 } } }
I20260928 14:41:55.461193 9839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:55.461400 9846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:41:55.461930 9846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:41:55.463428 9846 catalog_manager.cc:1347] Generated new cluster ID: 9c3075fdda764926b0c89bf1787e0612
I20260928 14:41:55.463482 9846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:41:55.483994 9846 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:41:55.484570 9846 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:41:55.495180 9846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e: Generated new TSK 0
I20260928 14:41:55.495416 9846 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:41:55.504245 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.65:0
--local_ip_for_outbound_sockets=127.22.13.65
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46747
--builtin_ntp_servers=127.22.13.84:44455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:41:55.591329 9857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:55.591550 9857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:55.591574 9857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:55.592824 9857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:55.592871 9857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:55.592905 9857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.65
I20260928 14:41:55.594553 9857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.13.65
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.65
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:55.594841 9857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:55.595166 9857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:55.600378 9865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:55.600384 9863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:55.600384 9862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:55.600670 9857 server_base.cc:1034] running on GCE node
I20260928 14:41:55.600917 9857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:55.601243 9857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:55.602538 9857 hybrid_clock.cc:648] HybridClock initialized: now 1790606515602517 us; error 40 us; skew 500 ppm
I20260928 14:41:55.603988 9857 webserver.cc:460] Webserver started at http://127.22.13.65:43549/ using document root <none> and password file <none>
I20260928 14:41:55.604259 9857 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:55.604332 9857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:55.604475 9857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:55.605618 9857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/data/instance:
uuid: "cf1cee34597943af948864f9f446b9a4"
format_stamp: "Formatted at 2026-09-28 14:41:55 on dist-test-slave-wwwb"
server_key: "a62b6156dceb5933aac112cfb08a4d0d"
server_key_iv: "302797b23f0fce0ee698208c2675c062"
server_key_version: "encryptionkey@0"
I20260928 14:41:55.606083 9857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance:
uuid: "cf1cee34597943af948864f9f446b9a4"
format_stamp: "Formatted at 2026-09-28 14:41:55 on dist-test-slave-wwwb"
server_key: "a62b6156dceb5933aac112cfb08a4d0d"
server_key_iv: "302797b23f0fce0ee698208c2675c062"
server_key_version: "encryptionkey@0"
I20260928 14:41:55.607529 9857 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:55.608340 9871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:55.608553 9857 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:55.608661 9857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/wal
uuid: "cf1cee34597943af948864f9f446b9a4"
format_stamp: "Formatted at 2026-09-28 14:41:55 on dist-test-slave-wwwb"
server_key: "a62b6156dceb5933aac112cfb08a4d0d"
server_key_iv: "302797b23f0fce0ee698208c2675c062"
server_key_version: "encryptionkey@0"
I20260928 14:41:55.608750 9857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:55.633190 9857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:55.633556 9857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:55.633714 9857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:55.634034 9857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:55.634392 9857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:55.634449 9857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:55.634482 9857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:55.634508 9857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:55.640223 9857 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.65:38775
I20260928 14:41:55.640317 9984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.65:38775 every 8 connection(s)
I20260928 14:41:55.640713 9857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/data/info.pb
I20260928 14:41:55.646548 9836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.641503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:38913 (local address 127.22.13.126:46747)
0928 14:41:55.641551 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.641553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.641785 (+ 232us) server_negotiation.cc:407] Connection header received
0928 14:41:55.642484 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.642487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.642506 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.642584 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:55.643356 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.643896 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.644945 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.645143 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.645224 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.646191 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:55.646192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:55.646192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:55.646198 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:55.646236 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:55.646245 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:55.646309 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.646419 (+ 110us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.646456 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:41:55.646770 9987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.641458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:46747 (local address 127.22.13.65:38913)
0928 14:41:55.641628 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:55.641679 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:55.642339 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:55.642665 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:55.642669 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:55.642680 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:55.643229 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.643234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.644004 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.644006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.644816 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.644818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.645234 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.645235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.645386 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.646041 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:41:55.646041 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:55.646071 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:41:55.646096 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:55.646311 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:55.646317 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:55.646321 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:41:55.646415 (+ 94us) client_negotiation.cc:240] Negotiation successful
0928 14:41:55.646530 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260928 14:41:55.647380 9985 heartbeater.cc:347] Connected to a master server at 127.22.13.126:46747
I20260928 14:41:55.647471 9985 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:55.647718 9985 heartbeater.cc:511] Master 127.22.13.126:46747 requested a full tablet report, sending...
I20260928 14:41:55.648277 9797 ts_manager.cc:194] Registered new tserver with Master: cf1cee34597943af948864f9f446b9a4 (127.22.13.65:38775)
I20260928 14:41:55.648865 9797 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.13.65:38913
I20260928 14:41:55.650936 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 9857
I20260928 14:41:55.651037 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-0/wal/instance
I20260928 14:41:55.651182 22581 external_mini_cluster.cc:1428] Setting key 8c014b7cf6c1731980eb38e59aa06727
I20260928 14:41:55.652074 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.66:0
--local_ip_for_outbound_sockets=127.22.13.66
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46747
--builtin_ntp_servers=127.22.13.84:44455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:41:55.735260 9988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:55.735487 9988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:55.735512 9988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:55.736622 9988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:55.736657 9988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:55.736682 9988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.66
I20260928 14:41:55.738083 9988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.13.66
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.66
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:55.738322 9988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:55.738596 9988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:55.743796 9996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:55.743796 9994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:55.743799 9993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:55.744015 9988 server_base.cc:1034] running on GCE node
I20260928 14:41:55.744343 9988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:55.744637 9988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:55.745808 9988 hybrid_clock.cc:648] HybridClock initialized: now 1790606515745794 us; error 47 us; skew 500 ppm
I20260928 14:41:55.747121 9988 webserver.cc:460] Webserver started at http://127.22.13.66:38919/ using document root <none> and password file <none>
I20260928 14:41:55.747388 9988 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:55.747455 9988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:55.747630 9988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:55.748762 9988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/data/instance:
uuid: "1f4a62c59f9e4bc7b5af0ca165031090"
format_stamp: "Formatted at 2026-09-28 14:41:55 on dist-test-slave-wwwb"
server_key: "07ff164dc0d7135a8fd3674ae7d05d56"
server_key_iv: "250feb0d6b3b9dc4ce6a5a4fd38ecf5a"
server_key_version: "encryptionkey@0"
I20260928 14:41:55.749145 9988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance:
uuid: "1f4a62c59f9e4bc7b5af0ca165031090"
format_stamp: "Formatted at 2026-09-28 14:41:55 on dist-test-slave-wwwb"
server_key: "07ff164dc0d7135a8fd3674ae7d05d56"
server_key_iv: "250feb0d6b3b9dc4ce6a5a4fd38ecf5a"
server_key_version: "encryptionkey@0"
I20260928 14:41:55.750676 9988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260928 14:41:55.751434 10002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:55.751639 9988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:55.751734 9988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/wal
uuid: "1f4a62c59f9e4bc7b5af0ca165031090"
format_stamp: "Formatted at 2026-09-28 14:41:55 on dist-test-slave-wwwb"
server_key: "07ff164dc0d7135a8fd3674ae7d05d56"
server_key_iv: "250feb0d6b3b9dc4ce6a5a4fd38ecf5a"
server_key_version: "encryptionkey@0"
I20260928 14:41:55.751814 9988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:55.768679 9988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:55.769037 9988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:55.769186 9988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:55.769448 9988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:55.769804 9988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:55.769860 9988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:55.769903 9988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:55.769958 9988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:55.776160 9988 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.66:34127
I20260928 14:41:55.776209 10115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.66:34127 every 8 connection(s)
I20260928 14:41:55.776574 9988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/data/info.pb
I20260928 14:41:55.777037 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 9988
I20260928 14:41:55.777172 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-1/wal/instance
I20260928 14:41:55.777395 22581 external_mini_cluster.cc:1428] Setting key 2dd53c67eafd3970a5f94d60cdfa777c
I20260928 14:41:55.778673 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.13.67:0
--local_ip_for_outbound_sockets=127.22.13.67
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46747
--builtin_ntp_servers=127.22.13.84:44455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 14:41:55.782905 9836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.777400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.66:50273 (local address 127.22.13.126:46747)
0928 14:41:55.777482 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.777485 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.777771 (+ 286us) server_negotiation.cc:407] Connection header received
0928 14:41:55.778559 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.778560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.778571 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.778616 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:55.779572 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.780173 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.781188 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.781418 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.781486 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.782551 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:55.782553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:55.782553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:55.782558 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:55.782574 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:55.782578 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:55.782656 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.782785 (+ 129us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.782817 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 14:41:55.783164 10118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.777383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:46747 (local address 127.22.13.66:50273)
0928 14:41:55.777646 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:55.777696 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:55.778446 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:55.778696 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:55.778701 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:55.778712 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:55.779467 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.779474 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.780285 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.780286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.781105 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.781107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.781535 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.781536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.781731 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.782373 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:41:55.782373 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:55.782416 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:41:55.782449 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:55.782661 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:55.782669 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:55.782673 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:41:55.782787 (+ 114us) client_negotiation.cc:240] Negotiation successful
0928 14:41:55.782910 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":123,"threads_started":1}
I20260928 14:41:55.783722 10116 heartbeater.cc:347] Connected to a master server at 127.22.13.126:46747
I20260928 14:41:55.783810 10116 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:55.784027 10116 heartbeater.cc:511] Master 127.22.13.126:46747 requested a full tablet report, sending...
I20260928 14:41:55.784461 9797 ts_manager.cc:194] Registered new tserver with Master: 1f4a62c59f9e4bc7b5af0ca165031090 (127.22.13.66:34127)
I20260928 14:41:55.785033 9797 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.13.66:50273
W20260928 14:41:55.863376 10119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:55.863610 10119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:55.863653 10119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:55.864904 10119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:55.864955 10119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:55.864996 10119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.13.67
I20260928 14:41:55.866451 10119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.13.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.13.67
--webserver_port=0
--tserver_master_addrs=127.22.13.126:46747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.13.67
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:55.866673 10119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:55.866948 10119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:55.872080 10125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:55.872201 10119 server_base.cc:1034] running on GCE node
W20260928 14:41:55.872090 10124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:55.872313 10127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:55.872601 10119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:55.872892 10119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:55.874013 10119 hybrid_clock.cc:648] HybridClock initialized: now 1790606515873987 us; error 35 us; skew 500 ppm
I20260928 14:41:55.875368 10119 webserver.cc:460] Webserver started at http://127.22.13.67:34575/ using document root <none> and password file <none>
I20260928 14:41:55.875631 10119 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:55.875680 10119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:55.875813 10119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:55.876926 10119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/data/instance:
uuid: "c41a922767a9403cae6bcc2be3c7db03"
format_stamp: "Formatted at 2026-09-28 14:41:55 on dist-test-slave-wwwb"
server_key: "b384be947f5a78191fce0cfb9acc7be3"
server_key_iv: "79d1e479f9867b8ca37e0bb414865b5b"
server_key_version: "encryptionkey@0"
I20260928 14:41:55.877310 10119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance:
uuid: "c41a922767a9403cae6bcc2be3c7db03"
format_stamp: "Formatted at 2026-09-28 14:41:55 on dist-test-slave-wwwb"
server_key: "b384be947f5a78191fce0cfb9acc7be3"
server_key_iv: "79d1e479f9867b8ca37e0bb414865b5b"
server_key_version: "encryptionkey@0"
I20260928 14:41:55.878796 10119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:41:55.879587 10133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:55.879781 10119 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:41:55.879923 10119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/wal
uuid: "c41a922767a9403cae6bcc2be3c7db03"
format_stamp: "Formatted at 2026-09-28 14:41:55 on dist-test-slave-wwwb"
server_key: "b384be947f5a78191fce0cfb9acc7be3"
server_key_iv: "79d1e479f9867b8ca37e0bb414865b5b"
server_key_version: "encryptionkey@0"
I20260928 14:41:55.880024 10119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:55.902456 10119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:55.902808 10119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:55.902964 10119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:55.903215 10119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:41:55.903558 10119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:41:55.903616 10119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:55.903663 10119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:41:55.903689 10119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:55.909643 10119 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.67:33417
I20260928 14:41:55.909690 10246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.67:33417 every 8 connection(s)
I20260928 14:41:55.910099 10119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/data/info.pb
I20260928 14:41:55.914742 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 10119
I20260928 14:41:55.914841 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/wal/instance
I20260928 14:41:55.915055 22581 external_mini_cluster.cc:1428] Setting key 99ae94be5570523335e426d1b0e651c9
I20260928 14:41:55.915812 9836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.910928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.67:33029 (local address 127.22.13.126:46747)
0928 14:41:55.911015 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.911018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.911252 (+ 234us) server_negotiation.cc:407] Connection header received
0928 14:41:55.911814 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.911815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.911830 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.911880 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:55.912649 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.913152 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.914199 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.914462 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.914539 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.915498 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:55.915499 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:55.915499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:55.915504 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:55.915522 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:55.915527 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:55.915585 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.915688 (+ 103us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.915718 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 14:41:55.916059 10249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.910883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:46747 (local address 127.22.13.67:33029)
0928 14:41:55.911129 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:55.911164 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:55.911694 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:55.911954 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:55.911957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:55.911965 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:41:55.912543 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.912548 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.913253 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.913254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.914113 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.914115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.914571 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.914573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.914710 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.915335 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:41:55.915335 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:41:55.915364 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:41:55.915406 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:41:55.915598 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:41:55.915604 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:41:55.915607 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:55.915684 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:41:55.915824 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":106,"threads_started":1}
I20260928 14:41:55.916517 10247 heartbeater.cc:347] Connected to a master server at 127.22.13.126:46747
I20260928 14:41:55.916632 10247 heartbeater.cc:464] Registering TS with master...
I20260928 14:41:55.916818 10247 heartbeater.cc:511] Master 127.22.13.126:46747 requested a full tablet report, sending...
I20260928 14:41:55.917192 9797 ts_manager.cc:194] Registered new tserver with Master: c41a922767a9403cae6bcc2be3c7db03 (127.22.13.67:33417)
I20260928 14:41:55.917572 9797 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.13.67:33029
I20260928 14:41:55.927121 22581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:41:55.931610 9836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.928630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48114 (local address 127.22.13.126:46747)
0928 14:41:55.928707 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.928708 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.928787 (+ 79us) server_negotiation.cc:407] Connection header received
0928 14:41:55.928831 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.928831 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.928841 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.928906 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:55.929063 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.929548 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.930429 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.930645 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.930697 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.931411 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:55.931412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:55.931413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:55.931417 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:55.931428 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:55.931432 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:55.931472 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.931516 (+ 44us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.931545 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 14:41:55.933182 9797 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:41:55.934892 9797 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:41:55.941599 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.937152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:34127 (local address 127.0.0.1:52318)
0928 14:41:55.937618 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:55.937630 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:55.937691 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:55.937979 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:55.937980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:55.937982 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:55.938036 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.938037 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.939365 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.939366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.940436 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.940438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.941264 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.941265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.941343 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.941357 (+ 14us) client_negotiation.cc:769] Sending connection context
0928 14:41:55.941441 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 14:41:55.941496 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":32,"thread_start_us":58,"threads_started":1}
I20260928 14:41:55.941718 10256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.937150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:33417 (local address 127.0.0.1:33058)
0928 14:41:55.937262 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:55.937300 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:55.937387 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:55.937908 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:55.937911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:55.937914 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:55.938022 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.938024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.939381 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.939382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.940647 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.940650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.941450 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.941451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.941516 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.941520 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:41:55.941585 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 14:41:55.941628 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":76,"thread_start_us":61,"threads_started":1}
I20260928 14:41:55.942003 10259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.937413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52318 (local address 127.22.13.66:34127)
0928 14:41:55.937597 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.937602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.937746 (+ 144us) server_negotiation.cc:407] Connection header received
0928 14:41:55.937810 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.937812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.937826 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.937890 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:55.938101 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.939264 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.940531 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.941174 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.941246 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.941336 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.941847 (+ 511us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.941897 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":110,"threads_started":1}
I20260928 14:41:55.942006 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.937410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33058 (local address 127.22.13.67:33417)
0928 14:41:55.937679 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.937683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.937693 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:41:55.937733 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.937735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.937752 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.937833 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:55.938121 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.939274 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.940744 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.941360 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.941436 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.941518 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.941600 (+ 82us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.941640 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":97,"threads_started":1}
I20260928 14:41:55.942140 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.937150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38775 (local address 127.0.0.1:52254)
0928 14:41:55.938135 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:55.938144 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:55.938186 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:55.938394 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:55.938395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:55.938396 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:55.938442 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.938443 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.939339 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.939340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.940809 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:55.940811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.941894 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.941895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:55.941976 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.941980 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:41:55.942025 (+ 45us) client_negotiation.cc:240] Negotiation successful
0928 14:41:55.942067 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":974,"mutex_wait_us":84,"thread_start_us":91,"threads_started":1}
I20260928 14:41:55.942521 10260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:55.937590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52254 (local address 127.22.13.65:38775)
0928 14:41:55.937742 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:55.937746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:55.938244 (+ 498us) server_negotiation.cc:407] Connection header received
0928 14:41:55.938292 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:55.938294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:55.938305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:55.938386 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:55.938520 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.939248 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.940894 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:55.941647 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:55.941894 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:55.942373 (+ 479us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:55.942396 (+ 23us) server_negotiation.cc:299] Negotiation successful
0928 14:41:55.942427 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260928 14:41:55.942755 10050 tablet_service.cc:1467] Processing CreateTablet for tablet 49bdf962189644d5a0e94e8e4f18dd15 (DEFAULT_TABLE table=test-table [id=72422754aa8040c283bcf1a4788df259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:55.942896 10181 tablet_service.cc:1467] Processing CreateTablet for tablet 49bdf962189644d5a0e94e8e4f18dd15 (DEFAULT_TABLE table=test-table [id=72422754aa8040c283bcf1a4788df259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:55.943142 10050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49bdf962189644d5a0e94e8e4f18dd15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:55.943176 9919 tablet_service.cc:1467] Processing CreateTablet for tablet 49bdf962189644d5a0e94e8e4f18dd15 (DEFAULT_TABLE table=test-table [id=72422754aa8040c283bcf1a4788df259]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:41:55.943285 10181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49bdf962189644d5a0e94e8e4f18dd15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:55.943476 9919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49bdf962189644d5a0e94e8e4f18dd15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:55.945930 10261 tablet_bootstrap.cc:492] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03: Bootstrap starting.
I20260928 14:41:55.946655 10262 tablet_bootstrap.cc:492] T 49bdf962189644d5a0e94e8e4f18dd15 P 1f4a62c59f9e4bc7b5af0ca165031090: Bootstrap starting.
I20260928 14:41:55.946698 10261 tablet_bootstrap.cc:654] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:55.947122 10261 log.cc:824] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:55.947281 10263 tablet_bootstrap.cc:492] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4: Bootstrap starting.
I20260928 14:41:55.947340 10262 tablet_bootstrap.cc:654] T 49bdf962189644d5a0e94e8e4f18dd15 P 1f4a62c59f9e4bc7b5af0ca165031090: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:55.947633 10262 log.cc:824] T 49bdf962189644d5a0e94e8e4f18dd15 P 1f4a62c59f9e4bc7b5af0ca165031090: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:55.947919 10263 tablet_bootstrap.cc:654] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:55.947989 10261 tablet_bootstrap.cc:492] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03: No bootstrap required, opened a new log
I20260928 14:41:55.948062 10261 ts_tablet_manager.cc:1397] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 14:41:55.948237 10263 log.cc:824] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:55.948377 10262 tablet_bootstrap.cc:492] T 49bdf962189644d5a0e94e8e4f18dd15 P 1f4a62c59f9e4bc7b5af0ca165031090: No bootstrap required, opened a new log
I20260928 14:41:55.948442 10262 ts_tablet_manager.cc:1397] T 49bdf962189644d5a0e94e8e4f18dd15 P 1f4a62c59f9e4bc7b5af0ca165031090: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:41:55.950363 10261 raft_consensus.cc:348] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41a922767a9403cae6bcc2be3c7db03" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33417 } } peers { permanent_uuid: "cf1cee34597943af948864f9f446b9a4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38775 } } peers { permanent_uuid: "1f4a62c59f9e4bc7b5af0ca165031090" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 34127 } }
I20260928 14:41:55.950547 10261 raft_consensus.cc:374] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:55.950587 10261 raft_consensus.cc:729] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c41a922767a9403cae6bcc2be3c7db03, State: Initialized, Role: FOLLOWER
I20260928 14:41:55.950619 10263 tablet_bootstrap.cc:492] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4: No bootstrap required, opened a new log
I20260928 14:41:55.950685 10263 ts_tablet_manager.cc:1397] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260928 14:41:55.950750 10261 consensus_queue.cc:260] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41a922767a9403cae6bcc2be3c7db03" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33417 } } peers { permanent_uuid: "cf1cee34597943af948864f9f446b9a4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38775 } } peers { permanent_uuid: "1f4a62c59f9e4bc7b5af0ca165031090" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 34127 } }
I20260928 14:41:55.950804 10262 raft_consensus.cc:348] T 49bdf962189644d5a0e94e8e4f18dd15 P 1f4a62c59f9e4bc7b5af0ca165031090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41a922767a9403cae6bcc2be3c7db03" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33417 } } peers { permanent_uuid: "cf1cee34597943af948864f9f446b9a4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38775 } } peers { permanent_uuid: "1f4a62c59f9e4bc7b5af0ca165031090" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 34127 } }
I20260928 14:41:55.950974 10262 raft_consensus.cc:374] T 49bdf962189644d5a0e94e8e4f18dd15 P 1f4a62c59f9e4bc7b5af0ca165031090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:55.951018 10262 raft_consensus.cc:729] T 49bdf962189644d5a0e94e8e4f18dd15 P 1f4a62c59f9e4bc7b5af0ca165031090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f4a62c59f9e4bc7b5af0ca165031090, State: Initialized, Role: FOLLOWER
I20260928 14:41:55.951032 10261 ts_tablet_manager.cc:1428] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260928 14:41:55.951133 10262 consensus_queue.cc:260] T 49bdf962189644d5a0e94e8e4f18dd15 P 1f4a62c59f9e4bc7b5af0ca165031090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41a922767a9403cae6bcc2be3c7db03" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33417 } } peers { permanent_uuid: "cf1cee34597943af948864f9f446b9a4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38775 } } peers { permanent_uuid: "1f4a62c59f9e4bc7b5af0ca165031090" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 34127 } }
I20260928 14:41:55.951174 10247 heartbeater.cc:503] Master 127.22.13.126:46747 was elected leader, sending a full tablet report...
I20260928 14:41:55.951378 10262 ts_tablet_manager.cc:1428] T 49bdf962189644d5a0e94e8e4f18dd15 P 1f4a62c59f9e4bc7b5af0ca165031090: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:41:55.951496 10116 heartbeater.cc:503] Master 127.22.13.126:46747 was elected leader, sending a full tablet report...
I20260928 14:41:55.953068 10263 raft_consensus.cc:348] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41a922767a9403cae6bcc2be3c7db03" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33417 } } peers { permanent_uuid: "cf1cee34597943af948864f9f446b9a4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38775 } } peers { permanent_uuid: "1f4a62c59f9e4bc7b5af0ca165031090" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 34127 } }
I20260928 14:41:55.953193 10263 raft_consensus.cc:374] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:55.953213 10263 raft_consensus.cc:729] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf1cee34597943af948864f9f446b9a4, State: Initialized, Role: FOLLOWER
I20260928 14:41:55.953343 10263 consensus_queue.cc:260] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41a922767a9403cae6bcc2be3c7db03" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33417 } } peers { permanent_uuid: "cf1cee34597943af948864f9f446b9a4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38775 } } peers { permanent_uuid: "1f4a62c59f9e4bc7b5af0ca165031090" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 34127 } }
I20260928 14:41:55.953612 10263 ts_tablet_manager.cc:1428] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:41:55.953691 9985 heartbeater.cc:503] Master 127.22.13.126:46747 was elected leader, sending a full tablet report...
W20260928 14:41:56.027225 10117 tablet.cc:2369] T 49bdf962189644d5a0e94e8e4f18dd15 P 1f4a62c59f9e4bc7b5af0ca165031090: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:41:56.034284 10269 raft_consensus.cc:482] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:41:56.034456 10269 raft_consensus.cc:504] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41a922767a9403cae6bcc2be3c7db03" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33417 } } peers { permanent_uuid: "cf1cee34597943af948864f9f446b9a4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38775 } } peers { permanent_uuid: "1f4a62c59f9e4bc7b5af0ca165031090" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 34127 } }
I20260928 14:41:56.034857 10269 leader_election.cc:288] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c41a922767a9403cae6bcc2be3c7db03 (127.22.13.67:33417), 1f4a62c59f9e4bc7b5af0ca165031090 (127.22.13.66:34127)
I20260928 14:41:56.037665 9987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:56.035035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.67:33417 (local address 127.22.13.65:49763)
0928 14:41:56.035133 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:56.035148 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:56.035196 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:56.035345 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:56.035346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:56.035348 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:56.035426 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:56.035428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:56.036085 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:56.036086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:56.036841 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:56.036843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:56.037401 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:56.037402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:56.037476 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:56.037479 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:56.037544 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 14:41:56.037589 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":5}
I20260928 14:41:56.037663 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:56.035115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:49763 (local address 127.22.13.67:33417)
0928 14:41:56.035162 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:56.035164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:56.035180 (+ 16us) server_negotiation.cc:407] Connection header received
0928 14:41:56.035260 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:56.035262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:56.035270 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:56.035343 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:56.035497 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:56.035996 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:56.036959 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:56.037317 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:56.037402 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:56.037422 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:56.037550 (+ 128us) server_negotiation.cc:299] Negotiation successful
0928 14:41:56.037589 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:41:56.037829 10259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:56.035164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.13.65:39553 (local address 127.22.13.66:34127)
0928 14:41:56.035204 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:56.035206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:56.035325 (+ 119us) server_negotiation.cc:407] Connection header received
0928 14:41:56.035394 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:56.035395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:56.035406 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:56.035482 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:41:56.035656 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:56.036121 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:56.037057 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:56.037423 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:56.037511 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:56.037518 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:56.037709 (+ 191us) server_negotiation.cc:299] Negotiation successful
0928 14:41:56.037750 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:41:56.037885 10270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:56.035078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:34127 (local address 127.22.13.65:39553)
0928 14:41:56.035259 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:56.035267 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:41:56.035340 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:41:56.035485 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:41:56.035486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:41:56.035487 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:41:56.035570 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:56.035572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:56.036209 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:56.036210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:56.036962 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:41:56.036964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:41:56.037512 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:41:56.037513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:41:56.037603 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:56.037606 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:41:56.037700 (+ 94us) client_negotiation.cc:240] Negotiation successful
0928 14:41:56.037739 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":94,"threads_started":1}
I20260928 14:41:56.038098 10201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49bdf962189644d5a0e94e8e4f18dd15" candidate_uuid: "cf1cee34597943af948864f9f446b9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c41a922767a9403cae6bcc2be3c7db03" is_pre_election: true
I20260928 14:41:56.038167 10070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49bdf962189644d5a0e94e8e4f18dd15" candidate_uuid: "cf1cee34597943af948864f9f446b9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f4a62c59f9e4bc7b5af0ca165031090" is_pre_election: true
I20260928 14:41:56.038257 10201 raft_consensus.cc:2454] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf1cee34597943af948864f9f446b9a4 in term 0.
I20260928 14:41:56.038332 10070 raft_consensus.cc:2454] T 49bdf962189644d5a0e94e8e4f18dd15 P 1f4a62c59f9e4bc7b5af0ca165031090 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf1cee34597943af948864f9f446b9a4 in term 0.
I20260928 14:41:56.038513 9873 leader_election.cc:302] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c41a922767a9403cae6bcc2be3c7db03, cf1cee34597943af948864f9f446b9a4; no voters:
I20260928 14:41:56.038637 10269 raft_consensus.cc:2781] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:41:56.038697 10269 raft_consensus.cc:482] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:41:56.038755 10269 raft_consensus.cc:3037] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:56.039503 10269 raft_consensus.cc:504] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41a922767a9403cae6bcc2be3c7db03" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33417 } } peers { permanent_uuid: "cf1cee34597943af948864f9f446b9a4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38775 } } peers { permanent_uuid: "1f4a62c59f9e4bc7b5af0ca165031090" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 34127 } }
I20260928 14:41:56.039656 10269 leader_election.cc:288] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [CANDIDATE]: Term 1 election: Requested vote from peers c41a922767a9403cae6bcc2be3c7db03 (127.22.13.67:33417), 1f4a62c59f9e4bc7b5af0ca165031090 (127.22.13.66:34127)
I20260928 14:41:56.039816 10201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49bdf962189644d5a0e94e8e4f18dd15" candidate_uuid: "cf1cee34597943af948864f9f446b9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c41a922767a9403cae6bcc2be3c7db03"
I20260928 14:41:56.039837 10070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49bdf962189644d5a0e94e8e4f18dd15" candidate_uuid: "cf1cee34597943af948864f9f446b9a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f4a62c59f9e4bc7b5af0ca165031090"
I20260928 14:41:56.039901 10201 raft_consensus.cc:3037] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:56.039928 10070 raft_consensus.cc:3037] T 49bdf962189644d5a0e94e8e4f18dd15 P 1f4a62c59f9e4bc7b5af0ca165031090 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:56.040643 10070 raft_consensus.cc:2454] T 49bdf962189644d5a0e94e8e4f18dd15 P 1f4a62c59f9e4bc7b5af0ca165031090 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf1cee34597943af948864f9f446b9a4 in term 1.
I20260928 14:41:56.040674 10201 raft_consensus.cc:2454] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf1cee34597943af948864f9f446b9a4 in term 1.
I20260928 14:41:56.040932 9875 leader_election.cc:302] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f4a62c59f9e4bc7b5af0ca165031090, cf1cee34597943af948864f9f446b9a4; no voters:
I20260928 14:41:56.041059 10269 raft_consensus.cc:2781] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:41:56.041285 10269 raft_consensus.cc:686] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [term 1 LEADER]: Becoming Leader. State: Replica: cf1cee34597943af948864f9f446b9a4, State: Running, Role: LEADER
I20260928 14:41:56.041400 10269 consensus_queue.cc:237] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41a922767a9403cae6bcc2be3c7db03" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33417 } } peers { permanent_uuid: "cf1cee34597943af948864f9f446b9a4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38775 } } peers { permanent_uuid: "1f4a62c59f9e4bc7b5af0ca165031090" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 34127 } }
I20260928 14:41:56.042361 9796 catalog_manager.cc:5495] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf1cee34597943af948864f9f446b9a4 (127.22.13.65). New cstate: current_term: 1 leader_uuid: "cf1cee34597943af948864f9f446b9a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c41a922767a9403cae6bcc2be3c7db03" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf1cee34597943af948864f9f446b9a4" member_type: VOTER last_known_addr { host: "127.22.13.65" port: 38775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f4a62c59f9e4bc7b5af0ca165031090" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 34127 } health_report { overall_health: UNKNOWN } } }
I20260928 14:41:56.061203 10260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:56.058507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52258 (local address 127.22.13.65:38775)
0928 14:41:56.058577 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:56.058579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:56.058592 (+ 13us) server_negotiation.cc:407] Connection header received
0928 14:41:56.058661 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:56.058662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:56.058686 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:56.058775 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:41:56.058910 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:56.059415 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:56.060232 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:56.060508 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:56.060582 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:56.060779 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:41:56.060941 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:41:56.061039 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:56.061073 (+ 34us) server_negotiation.cc:299] Negotiation successful
0928 14:41:56.061095 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:41:56.063153 10201 raft_consensus.cc:1261] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03 [term 1 FOLLOWER]: Refusing update from remote peer cf1cee34597943af948864f9f446b9a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:41:56.063169 10070 raft_consensus.cc:1261] T 49bdf962189644d5a0e94e8e4f18dd15 P 1f4a62c59f9e4bc7b5af0ca165031090 [term 1 FOLLOWER]: Refusing update from remote peer cf1cee34597943af948864f9f446b9a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:41:56.063520 10269 consensus_queue.cc:1035] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f4a62c59f9e4bc7b5af0ca165031090" member_type: VOTER last_known_addr { host: "127.22.13.66" port: 34127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:41:56.063669 10271 consensus_queue.cc:1035] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c41a922767a9403cae6bcc2be3c7db03" member_type: VOTER last_known_addr { host: "127.22.13.67" port: 33417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:41:56.066049 10274 mvcc.cc:205] Tried to move back new op lower bound from 7334324289791987712 to 7334324289706291200. Current Snapshot: MvccSnapshot[applied={T|T < 7334324289791987712}]
I20260928 14:41:56.066191 10275 mvcc.cc:205] Tried to move back new op lower bound from 7334324289791987712 to 7334324289706291200. Current Snapshot: MvccSnapshot[applied={T|T < 7334324289791987712}]
I20260928 14:41:56.068557 9795 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:41:56.068665 9795 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:41:56.069406 9795 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98f604f4c4f34fdab56d588b3dd8cc2e: Sending DeleteTablet for 3 replicas of tablet 49bdf962189644d5a0e94e8e4f18dd15
I20260928 14:41:56.069792 10181 tablet_service.cc:1514] Processing DeleteTablet for tablet 49bdf962189644d5a0e94e8e4f18dd15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:56 UTC) from {username='slave'} at 127.0.0.1:33058
I20260928 14:41:56.069799 10180 tablet_service.cc:1514] Processing DeleteTablet for tablet 49bdf962189644d5a0e94e8e4f18dd15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:56 UTC) from {username='slave'} at 127.0.0.1:33058
I20260928 14:41:56.069854 10050 tablet_service.cc:1514] Processing DeleteTablet for tablet 49bdf962189644d5a0e94e8e4f18dd15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:56 UTC) from {username='slave'} at 127.0.0.1:52318
I20260928 14:41:56.070128 9919 tablet_service.cc:1514] Processing DeleteTablet for tablet 49bdf962189644d5a0e94e8e4f18dd15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:41:56 UTC) from {username='slave'} at 127.0.0.1:52254
I20260928 14:41:56.070256 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 9857
I20260928 14:41:56.070386 10286 tablet_replica.cc:330] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4: stopping tablet replica
I20260928 14:41:56.070506 10286 raft_consensus.cc:2229] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:41:56.070660 10286 raft_consensus.cc:2258] T 49bdf962189644d5a0e94e8e4f18dd15 P cf1cee34597943af948864f9f446b9a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:56.071624 10284 tablet_replica.cc:330] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03: stopping tablet replica
I20260928 14:41:56.071746 10284 raft_consensus.cc:2229] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:41:56.071849 10284 raft_consensus.cc:2258] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:41:56.072220 10284 ts_tablet_manager.cc:1905] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:41:56.073566 10284 ts_tablet_manager.cc:1918] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:41:56.073657 10284 log.cc:1196] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03: Deleting WAL directory at /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606270465379-22581-0/minicluster-data/ts-2/wal/wals/49bdf962189644d5a0e94e8e4f18dd15
I20260928 14:41:56.073872 10284 ts_tablet_manager.cc:1939] T 49bdf962189644d5a0e94e8e4f18dd15 P c41a922767a9403cae6bcc2be3c7db03: Deleting consensus metadata
I20260928 14:41:56.074342 9782 catalog_manager.cc:4841] TS c41a922767a9403cae6bcc2be3c7db03 (127.22.13.67:33417): tablet 49bdf962189644d5a0e94e8e4f18dd15 (table test-table [id=72422754aa8040c283bcf1a4788df259]) successfully deleted
W20260928 14:41:56.074522 9782 catalog_manager.cc:4808] TS c41a922767a9403cae6bcc2be3c7db03 (127.22.13.67:33417): delete failed for tablet 49bdf962189644d5a0e94e8e4f18dd15 because the tablet was not found. No further retry: Not found: Tablet not found: 49bdf962189644d5a0e94e8e4f18dd15
W20260928 14:41:56.076644 9783 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.13.65:38775 (error 108)
I20260928 14:41:56.076851 10285 tablet_replica.cc:330] T 49bdf962189644d5a0e94e8e4f18dd15 P 1f4a62c59f9e4bc7b5af0ca165031090: stopping tablet replica
I20260928 14:41:56.076854 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 9988
I20260928 14:41:56.081619 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 10119
I20260928 14:41:56.081746 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:56.081560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.66:34127 (local address 127.0.0.1:52322)
0928 14:41:56.081636 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:56.081696 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.66:34127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260928 14:41:56.081749 10256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:56.081560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.65:38775 (local address 127.0.0.1:52272)
0928 14:41:56.081636 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0928 14:41:56.081683 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.65:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":26}
W20260928 14:41:56.082365 9783 catalog_manager.cc:4584] TS cf1cee34597943af948864f9f446b9a4 (127.22.13.65:38775): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 49bdf962189644d5a0e94e8e4f18dd15: Network error: Client connection negotiation failed: client connection to 127.22.13.65:38775: connect: Connection refused (error 111)
I20260928 14:41:56.086659 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 9766
2026-09-28T14:41:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (786 ms)
[----------] 6 tests from EncryptionPolicyTest (8051 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:41:56 dist-test-slave-wwwb krb5kdc[10290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:41:56 dist-test-slave-wwwb krb5kdc[10290](info): set up 2 sockets
Sep 28 14:41:56 dist-test-slave-wwwb krb5kdc[10290](info): commencing operation
krb5kdc: starting...
W20260928 14:41:57.118300 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:41:57 dist-test-slave-wwwb krb5kdc[10290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:41:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:41:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:41:57.237146 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37365
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:57.321168 10303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:57.321388 10303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:57.321413 10303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:57.322674 10303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:41:57.322724 10303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:57.322738 10303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:57.322751 10303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:41:57.322801 10303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:41:57.324877 10303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37365
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:57.325208 10303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:57.325490 10303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:57.330781 10309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:57.331019 10303 server_base.cc:1034] running on GCE node
W20260928 14:41:57.330792 10311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:57.330781 10308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:57.331372 10303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:57.331681 10303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:57.332829 10303 hybrid_clock.cc:648] HybridClock initialized: now 1790606517332819 us; error 28 us; skew 500 ppm
Sep 28 14:41:57 dist-test-slave-wwwb krb5kdc[10290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606517, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:57.339686 10303 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:41:57.340211 10303 webserver.cc:460] Webserver started at http://127.0.0.1:36253/ using document root <none> and password file <none>
I20260928 14:41:57.340462 10303 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:57.340515 10303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:57.340709 10303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:57.341858 10303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "cf00f98f3a1d4675ab238f2afb7877ef"
format_stamp: "Formatted at 2026-09-28 14:41:57 on dist-test-slave-wwwb"
server_key: "5fad6cb108705e5b598f2ed0ec046051"
server_key_iv: "fa6eed511685f992fe588c87d6039e43"
server_key_version: "encryptionkey@0"
I20260928 14:41:57.342293 10303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "cf00f98f3a1d4675ab238f2afb7877ef"
format_stamp: "Formatted at 2026-09-28 14:41:57 on dist-test-slave-wwwb"
server_key: "5fad6cb108705e5b598f2ed0ec046051"
server_key_iv: "fa6eed511685f992fe588c87d6039e43"
server_key_version: "encryptionkey@0"
I20260928 14:41:57.343621 10303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:57.344337 10318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:57.344523 10303 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:57.344635 10303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "cf00f98f3a1d4675ab238f2afb7877ef"
format_stamp: "Formatted at 2026-09-28 14:41:57 on dist-test-slave-wwwb"
server_key: "5fad6cb108705e5b598f2ed0ec046051"
server_key_iv: "fa6eed511685f992fe588c87d6039e43"
server_key_version: "encryptionkey@0"
I20260928 14:41:57.344713 10303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:57.366699 10303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:57.368691 10303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:57.368867 10303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:57.372996 10303 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37365
I20260928 14:41:57.373045 10370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37365 every 8 connection(s)
I20260928 14:41:57.373433 10303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:41:57.374369 10371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:57.376749 10371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef: Bootstrap starting.
I20260928 14:41:57.377316 10371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:57.377568 10371 log.cc:824] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:57.378320 10371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef: No bootstrap required, opened a new log
I20260928 14:41:57.379916 10371 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf00f98f3a1d4675ab238f2afb7877ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37365 } }
I20260928 14:41:57.380064 10371 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:57.380087 10371 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf00f98f3a1d4675ab238f2afb7877ef, State: Initialized, Role: FOLLOWER
I20260928 14:41:57.380239 10371 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf00f98f3a1d4675ab238f2afb7877ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37365 } }
I20260928 14:41:57.380326 10371 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:41:57.380350 10371 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:41:57.380380 10371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:57.380916 10371 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf00f98f3a1d4675ab238f2afb7877ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37365 } }
I20260928 14:41:57.381001 10371 leader_election.cc:302] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf00f98f3a1d4675ab238f2afb7877ef; no voters:
I20260928 14:41:57.381127 10371 leader_election.cc:288] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:41:57.381203 10374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:41:57.381376 10374 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [term 1 LEADER]: Becoming Leader. State: Replica: cf00f98f3a1d4675ab238f2afb7877ef, State: Running, Role: LEADER
I20260928 14:41:57.381405 10371 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:41:57.381466 10374 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf00f98f3a1d4675ab238f2afb7877ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37365 } }
I20260928 14:41:57.381861 10376 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf00f98f3a1d4675ab238f2afb7877ef. Latest consensus state: current_term: 1 leader_uuid: "cf00f98f3a1d4675ab238f2afb7877ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf00f98f3a1d4675ab238f2afb7877ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37365 } } }
I20260928 14:41:57.381961 10376 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:57.382093 10375 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf00f98f3a1d4675ab238f2afb7877ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf00f98f3a1d4675ab238f2afb7877ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37365 } } }
I20260928 14:41:57.382162 10375 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:57.382464 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 10303
I20260928 14:41:57.382318 10381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:41:57.382587 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:41:57.382772 22581 external_mini_cluster.cc:1428] Setting key 7587469b225a747173a504fac62e4a7b
I20260928 14:41:57.383394 10381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:41:57.385000 10381 catalog_manager.cc:1347] Generated new cluster ID: 3e85be09b92a4ae98d76aa1c6082c4f3
I20260928 14:41:57.385054 10381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:41:57 dist-test-slave-wwwb krb5kdc[10290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 14:41:57.393043 10381 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:41:57.393709 10381 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:41:57.395258 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:57.383323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37593 (local address 127.0.0.1:37365)
0928 14:41:57.383521 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:57.383532 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:57.383573 (+ 41us) server_negotiation.cc:407] Connection header received
0928 14:41:57.383694 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:57.383700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:57.383756 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:57.383944 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:57.384758 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:57.386237 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:57.387412 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:57.388050 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:57.388157 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:57.391135 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:57.391141 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:57.391142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:57.391159 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:57.393323 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:57.393884 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:57.393888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:57.393888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:57.394029 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:57.394299 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:57.394300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:57.394301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:57.394627 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:57.394803 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:57.394915 (+ 112us) server_negotiation.cc:299] Negotiation successful
0928 14:41:57.395024 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260928 14:41:57.396250 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:41:57.400239 10381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf00f98f3a1d4675ab238f2afb7877ef: Generated new TSK 0
I20260928 14:41:57.400454 10381 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:41:57.407164 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:57.400018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36871 (local address 127.0.0.1:37365)
0928 14:41:57.400080 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:57.400082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:57.400292 (+ 210us) server_negotiation.cc:407] Connection header received
0928 14:41:57.400316 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:57.400317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:57.400327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:57.400371 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:57.401088 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:57.401602 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:57.402703 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:57.402892 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:57.402944 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:57.404404 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:57.404405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:57.404405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:41:57.404411 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:57.405852 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:57.406370 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:57.406373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:57.406374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:57.406432 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:41:57.406668 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:41:57.406668 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:41:57.406669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:41:57.406771 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:57.406861 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:57.407023 (+ 162us) server_negotiation.cc:299] Negotiation successful
0928 14:41:57.407071 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 14:41:57.409363 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 10303
W20260928 14:41:57.413571 10288 connection.cc:535] client connection to 127.0.0.1:37365 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37365): Connection reset by peer (error 104)
2026-09-28T14:41:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:41:57 dist-test-slave-wwwb krb5kdc[10402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:41:57 dist-test-slave-wwwb krb5kdc[10402](info): set up 2 sockets
Sep 28 14:41:57 dist-test-slave-wwwb krb5kdc[10402](info): commencing operation
krb5kdc: starting...
W20260928 14:41:58.422326 22581 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:41:58 dist-test-slave-wwwb krb5kdc[10402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:41:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:41:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:41:58.541610 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37575
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:58.622802 10415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:58.623050 10415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:58.623090 10415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:58.624256 10415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:41:58.624331 10415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:58.624358 10415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:58.624390 10415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:41:58.624418 10415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:41:58.626027 10415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37575
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:58.626320 10415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:58.626566 10415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:58.631614 10420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:58.631614 10423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:58.631680 10421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:58.631955 10415 server_base.cc:1034] running on GCE node
I20260928 14:41:58.632196 10415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:58.632516 10415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:58.633670 10415 hybrid_clock.cc:648] HybridClock initialized: now 1790606518633647 us; error 39 us; skew 500 ppm
Sep 28 14:41:58 dist-test-slave-wwwb krb5kdc[10402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606518, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:58.636276 10415 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:41:58.636766 10415 webserver.cc:460] Webserver started at http://127.0.0.1:36577/ using document root <none> and password file <none>
I20260928 14:41:58.636950 10415 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:58.636988 10415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:58.637080 10415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:58.638185 10415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "b9ef355110d740b78a4dc238c80b2514"
format_stamp: "Formatted at 2026-09-28 14:41:58 on dist-test-slave-wwwb"
server_key: "0d34d9c37ed35e323c0049544a279582"
server_key_iv: "d4954644071738c0fefcf2a33a7671ac"
server_key_version: "encryptionkey@0"
I20260928 14:41:58.638489 10415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "b9ef355110d740b78a4dc238c80b2514"
format_stamp: "Formatted at 2026-09-28 14:41:58 on dist-test-slave-wwwb"
server_key: "0d34d9c37ed35e323c0049544a279582"
server_key_iv: "d4954644071738c0fefcf2a33a7671ac"
server_key_version: "encryptionkey@0"
I20260928 14:41:58.639803 10415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:58.640520 10430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:58.640731 10415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:41:58.640831 10415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "b9ef355110d740b78a4dc238c80b2514"
format_stamp: "Formatted at 2026-09-28 14:41:58 on dist-test-slave-wwwb"
server_key: "0d34d9c37ed35e323c0049544a279582"
server_key_iv: "d4954644071738c0fefcf2a33a7671ac"
server_key_version: "encryptionkey@0"
I20260928 14:41:58.640918 10415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:58.663149 10415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:58.665652 10415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:58.665772 10415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:58.669577 10415 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37575
I20260928 14:41:58.669706 10482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37575 every 8 connection(s)
I20260928 14:41:58.670094 10415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:41:58.671027 10483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:58.673331 10483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514: Bootstrap starting.
I20260928 14:41:58.673928 10483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:58.674300 10483 log.cc:824] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:58.675030 10483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514: No bootstrap required, opened a new log
I20260928 14:41:58.676688 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 10415
I20260928 14:41:58.676829 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:41:58.676717 10483 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ef355110d740b78a4dc238c80b2514" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37575 } }
I20260928 14:41:58.676910 10483 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:58.676949 10483 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ef355110d740b78a4dc238c80b2514, State: Initialized, Role: FOLLOWER
I20260928 14:41:58.677037 22581 external_mini_cluster.cc:1428] Setting key 271ef3e954f97418162a637e600dbfa8
I20260928 14:41:58.677107 10483 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ef355110d740b78a4dc238c80b2514" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37575 } }
I20260928 14:41:58.677198 10483 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:41:58.677237 10483 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:41:58.677268 10483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:58.678414 10483 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ef355110d740b78a4dc238c80b2514" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37575 } }
I20260928 14:41:58.678524 10483 leader_election.cc:302] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9ef355110d740b78a4dc238c80b2514; no voters:
I20260928 14:41:58.678686 10483 leader_election.cc:288] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:41:58.678761 10488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:41:58.678901 10488 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [term 1 LEADER]: Becoming Leader. State: Replica: b9ef355110d740b78a4dc238c80b2514, State: Running, Role: LEADER
I20260928 14:41:58.678926 10483 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:41:58.678984 10488 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ef355110d740b78a4dc238c80b2514" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37575 } }
I20260928 14:41:58.679313 10489 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9ef355110d740b78a4dc238c80b2514" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ef355110d740b78a4dc238c80b2514" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37575 } } }
I20260928 14:41:58.679414 10489 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:58.679303 10488 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9ef355110d740b78a4dc238c80b2514. Latest consensus state: current_term: 1 leader_uuid: "b9ef355110d740b78a4dc238c80b2514" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9ef355110d740b78a4dc238c80b2514" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37575 } } }
I20260928 14:41:58.679589 10488 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:58.679746 10496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:41:58.680447 10496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:41:58.682267 10496 catalog_manager.cc:1347] Generated new cluster ID: 0a99be0850444cea80e567db2d259071
I20260928 14:41:58.682356 10496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:41:58.683259 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:58.677538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58881 (local address 127.0.0.1:37575)
0928 14:41:58.677737 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:58.677750 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:58.677784 (+ 34us) server_negotiation.cc:407] Connection header received
0928 14:41:58.677954 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:58.677963 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:58.678038 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:58.678226 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:58.678416 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:58.679867 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:58.681098 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:58.681683 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:58.681805 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:58.682627 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:58.682631 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:58.682631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:58.682797 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:58.682830 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:58.682840 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:58.682942 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:58.683060 (+ 118us) server_negotiation.cc:299] Negotiation successful
0928 14:41:58.683088 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260928 14:41:58.684315 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:41:58.689008 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:58.685751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36951 (local address 127.0.0.1:37575)
0928 14:41:58.685802 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:58.685805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:58.686103 (+ 298us) server_negotiation.cc:407] Connection header received
0928 14:41:58.686172 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:58.686173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:58.686183 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:58.686240 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:58.686391 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:58.686899 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:58.687639 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:41:58.687910 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:41:58.687999 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:41:58.688724 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:58.688726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:58.688726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:58.688732 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:58.688748 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:58.688752 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:58.688826 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:58.688888 (+ 62us) server_negotiation.cc:299] Negotiation successful
0928 14:41:58.688901 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:41:58.712772 10496 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:41:58.713500 10496 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:41:58.717353 10496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9ef355110d740b78a4dc238c80b2514: Generated new TSK 0
I20260928 14:41:58.717602 10496 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:41:58.721563 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 10415
W20260928 14:41:58.726154 10400 connection.cc:535] client connection to 127.0.0.1:37575 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37575): Connection reset by peer (error 104)
2026-09-28T14:41:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1312 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:41:58 dist-test-slave-wwwb krb5kdc[10514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:41:58 dist-test-slave-wwwb krb5kdc[10514](info): set up 2 sockets
Sep 28 14:41:58 dist-test-slave-wwwb krb5kdc[10514](info): commencing operation
krb5kdc: starting...
W20260928 14:41:59.750232 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:41:59 dist-test-slave-wwwb krb5kdc[10514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:41:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:41:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:41:59.869206 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:41:59.951979 10527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:41:59.952258 10527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:41:59.952302 10527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:41:59.953573 10527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:41:59.953660 10527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:41:59.953685 10527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:41:59.953715 10527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:41:59.953747 10527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:41:59.955400 10527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39695
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:41:59.955713 10527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:41:59.955956 10527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:41:59.961033 10535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:59.961033 10533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:41:59.961050 10532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:41:59.962152 10527 server_base.cc:1034] running on GCE node
I20260928 14:41:59.962397 10527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:41:59.962692 10527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:41:59.963850 10527 hybrid_clock.cc:648] HybridClock initialized: now 1790606519963836 us; error 45 us; skew 500 ppm
Sep 28 14:41:59 dist-test-slave-wwwb krb5kdc[10514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606519, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:41:59.966423 10527 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:41:59.966885 10527 webserver.cc:460] Webserver started at http://127.0.0.1:36903/ using document root <none> and password file <none>
I20260928 14:41:59.967123 10527 fs_manager.cc:359] Metadata directory not provided
I20260928 14:41:59.967170 10527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:41:59.967337 10527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:41:59.968431 10527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "f0c6a6dce71a4b4f8f6598e45579b588"
format_stamp: "Formatted at 2026-09-28 14:41:59 on dist-test-slave-wwwb"
server_key: "30838604075da71066bf2c7c752cdb3f"
server_key_iv: "17fd09e245949553e6497c3d6e7770dc"
server_key_version: "encryptionkey@0"
I20260928 14:41:59.968802 10527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "f0c6a6dce71a4b4f8f6598e45579b588"
format_stamp: "Formatted at 2026-09-28 14:41:59 on dist-test-slave-wwwb"
server_key: "30838604075da71066bf2c7c752cdb3f"
server_key_iv: "17fd09e245949553e6497c3d6e7770dc"
server_key_version: "encryptionkey@0"
I20260928 14:41:59.970258 10527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 14:41:59.971028 10542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:41:59.971215 10527 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:41:59.971303 10527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "f0c6a6dce71a4b4f8f6598e45579b588"
format_stamp: "Formatted at 2026-09-28 14:41:59 on dist-test-slave-wwwb"
server_key: "30838604075da71066bf2c7c752cdb3f"
server_key_iv: "17fd09e245949553e6497c3d6e7770dc"
server_key_version: "encryptionkey@0"
I20260928 14:41:59.971379 10527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:41:59.974888 10527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:41:59.976790 10527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:41:59.976958 10527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:41:59.980959 10527 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39695
I20260928 14:41:59.981009 10594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39695 every 8 connection(s)
I20260928 14:41:59.981402 10527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:41:59.982501 10595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:41:59.984086 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 10527
I20260928 14:41:59.984213 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:41:59.984390 22581 external_mini_cluster.cc:1428] Setting key 1aa9ac2e2d778d3a4c9506565f06f115
I20260928 14:41:59.985255 10595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588: Bootstrap starting.
I20260928 14:41:59.986045 10595 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588: Neither blocks nor log segments found. Creating new log.
I20260928 14:41:59.986392 10595 log.cc:824] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588: Log is configured to *not* fsync() on all Append() calls
I20260928 14:41:59.987140 10595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588: No bootstrap required, opened a new log
I20260928 14:41:59.987223 10598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:59.984947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49953 (local address 127.0.0.1:39695)
0928 14:41:59.985140 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:59.985149 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:59.985182 (+ 33us) server_negotiation.cc:407] Connection header received
0928 14:41:59.985780 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:59.985789 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:59.985822 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:59.986016 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:59.986771 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:59.986775 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:59.986775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:59.986780 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:59.986816 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:59.986826 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:59.986927 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:59.986993 (+ 66us) server_negotiation.cc:299] Negotiation successful
0928 14:41:59.987036 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":80,"threads_started":1}
I20260928 14:41:59.988202 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:41:59.989454 10595 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c6a6dce71a4b4f8f6598e45579b588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39695 } }
I20260928 14:41:59.989639 10595 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:41:59.989686 10595 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0c6a6dce71a4b4f8f6598e45579b588, State: Initialized, Role: FOLLOWER
I20260928 14:41:59.989823 10595 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c6a6dce71a4b4f8f6598e45579b588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39695 } }
I20260928 14:41:59.989971 10595 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:41:59.990021 10595 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:41:59.990080 10595 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:41:59.990896 10598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:41:59.989677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36501 (local address 127.0.0.1:39695)
0928 14:41:59.989718 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 14:41:59.989720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:41:59.989895 (+ 175us) server_negotiation.cc:407] Connection header received
0928 14:41:59.989996 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:41:59.989997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:41:59.990002 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:41:59.990065 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:41:59.990650 (+ 585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:41:59.990651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:41:59.990651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:41:59.990654 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:41:59.990668 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:41:59.990672 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:41:59.990717 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 14:41:59.990798 (+ 81us) server_negotiation.cc:299] Negotiation successful
0928 14:41:59.990810 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:41:59.990955 10595 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c6a6dce71a4b4f8f6598e45579b588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39695 } }
I20260928 14:41:59.991102 10595 leader_election.cc:302] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0c6a6dce71a4b4f8f6598e45579b588; no voters:
I20260928 14:41:59.991295 10595 leader_election.cc:288] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:41:59.991317 10605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:41:59.991438 10605 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [term 1 LEADER]: Becoming Leader. State: Replica: f0c6a6dce71a4b4f8f6598e45579b588, State: Running, Role: LEADER
I20260928 14:41:59.991529 10605 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c6a6dce71a4b4f8f6598e45579b588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39695 } }
I20260928 14:41:59.991559 10595 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:41:59.993639 10606 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0c6a6dce71a4b4f8f6598e45579b588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c6a6dce71a4b4f8f6598e45579b588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39695 } } }
I20260928 14:41:59.993697 10606 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [sys.catalog]: This master's current role is: LEADER
I20260928 14:41:59.993479 10607 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0c6a6dce71a4b4f8f6598e45579b588. Latest consensus state: current_term: 1 leader_uuid: "f0c6a6dce71a4b4f8f6598e45579b588" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0c6a6dce71a4b4f8f6598e45579b588" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39695 } } }
I20260928 14:41:59.993848 10607 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588 [sys.catalog]: This master's current role is: LEADER
W20260928 14:41:59.993973 10619 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:41:59.994038 10619 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:41:59.994122 10621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:41:59.994307 10621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:41:59.995826 10621 catalog_manager.cc:1347] Generated new cluster ID: ba0f58670be84519a1ee28e049569e5a
I20260928 14:41:59.995886 10621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:42:00.011898 10621 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:00.012126 10621 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:00.024044 10621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0c6a6dce71a4b4f8f6598e45579b588: Generated new TSK 0
I20260928 14:42:00.024292 10621 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:00.027463 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 10527
2026-09-28T14:42:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:00 dist-test-slave-wwwb krb5kdc[10626](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:00 dist-test-slave-wwwb krb5kdc[10626](info): set up 2 sockets
Sep 28 14:42:00 dist-test-slave-wwwb krb5kdc[10626](info): commencing operation
krb5kdc: starting...
W20260928 14:42:01.042375 22581 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:01 dist-test-slave-wwwb krb5kdc[10626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:42:01.160310 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33145
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:01.240655 10639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:01.240926 10639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:01.240967 10639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:01.242173 10639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:01.242252 10639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:01.242277 10639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:01.242304 10639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:01.242333 10639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:01.243829 10639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:01.244140 10639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:01.244383 10639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:01.249369 10647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:01.249372 10645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:01.249400 10639 server_base.cc:1034] running on GCE node
W20260928 14:42:01.249401 10644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:01.249795 10639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:01.250139 10639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:01.251309 10639 hybrid_clock.cc:648] HybridClock initialized: now 1790606521251305 us; error 35 us; skew 500 ppm
Sep 28 14:42:01 dist-test-slave-wwwb krb5kdc[10626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606521, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:01.253693 10639 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:42:01.254201 10639 webserver.cc:460] Webserver started at http://127.0.0.1:39183/ using document root <none> and password file <none>
I20260928 14:42:01.254432 10639 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:01.254503 10639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:01.254665 10639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:01.255779 10639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "602987af694044ac8a8623b976e99ba7"
format_stamp: "Formatted at 2026-09-28 14:42:01 on dist-test-slave-wwwb"
server_key: "0847b6990cc038a850cc58b8b85f9d3d"
server_key_iv: "ce50b1b8504fe94be44c99b4291804fd"
server_key_version: "encryptionkey@0"
I20260928 14:42:01.256155 10639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "602987af694044ac8a8623b976e99ba7"
format_stamp: "Formatted at 2026-09-28 14:42:01 on dist-test-slave-wwwb"
server_key: "0847b6990cc038a850cc58b8b85f9d3d"
server_key_iv: "ce50b1b8504fe94be44c99b4291804fd"
server_key_version: "encryptionkey@0"
I20260928 14:42:01.257532 10639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:42:01.258363 10654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:01.258527 10639 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:01.258646 10639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "602987af694044ac8a8623b976e99ba7"
format_stamp: "Formatted at 2026-09-28 14:42:01 on dist-test-slave-wwwb"
server_key: "0847b6990cc038a850cc58b8b85f9d3d"
server_key_iv: "ce50b1b8504fe94be44c99b4291804fd"
server_key_version: "encryptionkey@0"
I20260928 14:42:01.258719 10639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:01.302949 10639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:01.304812 10639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:01.305016 10639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:01.309137 10639 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33145
I20260928 14:42:01.309209 10706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33145 every 8 connection(s)
I20260928 14:42:01.309578 10639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:01.310524 10707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:01.312813 10707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7: Bootstrap starting.
I20260928 14:42:01.313410 10707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:01.313690 10707 log.cc:824] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:01.314397 10707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7: No bootstrap required, opened a new log
I20260928 14:42:01.316026 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 10639
I20260928 14:42:01.316159 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:01.316022 10707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602987af694044ac8a8623b976e99ba7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33145 } }
I20260928 14:42:01.316224 10707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:01.316263 10707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602987af694044ac8a8623b976e99ba7, State: Initialized, Role: FOLLOWER
I20260928 14:42:01.316373 22581 external_mini_cluster.cc:1428] Setting key 226d9cb326ea12827ae672929275b717
I20260928 14:42:01.316435 10707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602987af694044ac8a8623b976e99ba7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33145 } }
I20260928 14:42:01.316520 10707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:01.316565 10707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:01.316612 10707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:01.317550 10707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602987af694044ac8a8623b976e99ba7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33145 } }
I20260928 14:42:01.317662 10707 leader_election.cc:302] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 602987af694044ac8a8623b976e99ba7; no voters:
I20260928 14:42:01.317838 10707 leader_election.cc:288] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:01.317894 10712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:01.318121 10707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:01.318157 10712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [term 1 LEADER]: Becoming Leader. State: Replica: 602987af694044ac8a8623b976e99ba7, State: Running, Role: LEADER
I20260928 14:42:01.319540 10712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602987af694044ac8a8623b976e99ba7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33145 } }
I20260928 14:42:01.320190 10714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 602987af694044ac8a8623b976e99ba7. Latest consensus state: current_term: 1 leader_uuid: "602987af694044ac8a8623b976e99ba7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602987af694044ac8a8623b976e99ba7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33145 } } }
I20260928 14:42:01.320199 10713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "602987af694044ac8a8623b976e99ba7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "602987af694044ac8a8623b976e99ba7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33145 } } }
I20260928 14:42:01.320281 10714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:01.320298 10713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:01.320518 10726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:01.321019 10726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:01.322791 10726 catalog_manager.cc:1347] Generated new cluster ID: 9a06a7aefbc04013b86ba786fb63fe68
I20260928 14:42:01.322845 10726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:42:01 dist-test-slave-wwwb krb5kdc[10626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 14:42:01.327728 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:01.316858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45341 (local address 127.0.0.1:33145)
0928 14:42:01.317046 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:01.317057 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:01.317097 (+ 40us) server_negotiation.cc:407] Connection header received
0928 14:42:01.317217 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:01.317222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:01.317263 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:01.317400 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:01.318330 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:01.319802 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:01.320892 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:01.321536 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:01.321683 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:01.324299 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:01.324302 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:01.324303 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:01.324315 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:01.326100 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:01.326485 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:01.326488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:01.326489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:01.326600 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:01.326816 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:01.326817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:01.326817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:01.327074 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:01.327258 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:01.327369 (+ 111us) server_negotiation.cc:299] Negotiation successful
0928 14:42:01.327515 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260928 14:42:01.328722 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:01.334612 10726 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:01.335333 10726 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:01.340713 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:01.330192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34011 (local address 127.0.0.1:33145)
0928 14:42:01.330230 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:01.330232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:01.334154 (+ 3922us) server_negotiation.cc:407] Connection header received
0928 14:42:01.334218 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:01.334219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:01.334232 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:01.334291 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:01.335039 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:01.335584 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:01.336303 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:01.336525 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:01.336603 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:01.337879 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:01.337880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:01.337880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:01.337888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:01.339380 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:01.339752 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:01.339754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:01.339755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:01.339814 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:01.340063 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:01.340064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:01.340065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:01.340195 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:01.340279 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:01.340491 (+ 212us) server_negotiation.cc:299] Negotiation successful
0928 14:42:01.340625 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":18}
I20260928 14:42:01.344986 10726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 602987af694044ac8a8623b976e99ba7: Generated new TSK 0
I20260928 14:42:01.345250 10726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:01.350965 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 10639
2026-09-28T14:42:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1323 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:01 dist-test-slave-wwwb krb5kdc[10738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:01 dist-test-slave-wwwb krb5kdc[10738](info): set up 2 sockets
Sep 28 14:42:01 dist-test-slave-wwwb krb5kdc[10738](info): commencing operation
krb5kdc: starting...
W20260928 14:42:02.386277 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:02 dist-test-slave-wwwb krb5kdc[10738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:42:02.505275 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38131
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:02.587550 10751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:02.587865 10751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:02.587914 10751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:02.589217 10751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:02.589304 10751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:02.589329 10751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:02.589350 10751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:02.589380 10751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:02.591032 10751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:02.591369 10751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:02.591609 10751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:02.596606 10759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:02.596606 10756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:02.596599 10757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:02.596968 10751 server_base.cc:1034] running on GCE node
I20260928 14:42:02.597241 10751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:02.597554 10751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:02.598706 10751 hybrid_clock.cc:648] HybridClock initialized: now 1790606522598681 us; error 37 us; skew 500 ppm
Sep 28 14:42:02 dist-test-slave-wwwb krb5kdc[10738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606522, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:02.606303 10751 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:42:02.606856 10751 webserver.cc:460] Webserver started at http://127.0.0.1:39137/ using document root <none> and password file <none>
I20260928 14:42:02.607090 10751 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:02.607165 10751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:02.607336 10751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:02.608484 10751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "afd12b76bb524d0a907f0121ca68ae42"
format_stamp: "Formatted at 2026-09-28 14:42:02 on dist-test-slave-wwwb"
server_key: "6b3ca280575b3fcd4a0d4ec5964c8072"
server_key_iv: "719780f040dc30778569adc9717322ca"
server_key_version: "encryptionkey@0"
I20260928 14:42:02.608870 10751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "afd12b76bb524d0a907f0121ca68ae42"
format_stamp: "Formatted at 2026-09-28 14:42:02 on dist-test-slave-wwwb"
server_key: "6b3ca280575b3fcd4a0d4ec5964c8072"
server_key_iv: "719780f040dc30778569adc9717322ca"
server_key_version: "encryptionkey@0"
I20260928 14:42:02.610409 10751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:42:02.611171 10766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:02.611348 10751 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:42:02.611459 10751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "afd12b76bb524d0a907f0121ca68ae42"
format_stamp: "Formatted at 2026-09-28 14:42:02 on dist-test-slave-wwwb"
server_key: "6b3ca280575b3fcd4a0d4ec5964c8072"
server_key_iv: "719780f040dc30778569adc9717322ca"
server_key_version: "encryptionkey@0"
I20260928 14:42:02.611536 10751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:02.628017 10751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:02.630066 10751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:02.630241 10751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:02.634778 10751 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38131
I20260928 14:42:02.634843 10818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38131 every 8 connection(s)
I20260928 14:42:02.635237 10751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:02.636138 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:02.638532 10819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42: Bootstrap starting.
I20260928 14:42:02.639132 10819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:02.639380 10819 log.cc:824] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:02.640108 10819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42: No bootstrap required, opened a new log
I20260928 14:42:02.640759 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 10751
I20260928 14:42:02.640903 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:02.641124 22581 external_mini_cluster.cc:1428] Setting key 411688aa7d7115e7602764efbc66aa58
I20260928 14:42:02.642561 10819 raft_consensus.cc:348] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afd12b76bb524d0a907f0121ca68ae42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } }
I20260928 14:42:02.642761 10819 raft_consensus.cc:374] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:02.642805 10819 raft_consensus.cc:729] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afd12b76bb524d0a907f0121ca68ae42, State: Initialized, Role: FOLLOWER
I20260928 14:42:02.643002 10819 consensus_queue.cc:260] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afd12b76bb524d0a907f0121ca68ae42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } }
I20260928 14:42:02.643182 10819 raft_consensus.cc:388] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:02.643226 10819 raft_consensus.cc:482] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:02.643316 10819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:02.643985 10819 raft_consensus.cc:504] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afd12b76bb524d0a907f0121ca68ae42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } }
I20260928 14:42:02.644078 10819 leader_election.cc:302] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afd12b76bb524d0a907f0121ca68ae42; no voters:
I20260928 14:42:02.644279 10819 leader_election.cc:288] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:02.644301 10824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:02.644424 10824 raft_consensus.cc:686] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [term 1 LEADER]: Becoming Leader. State: Replica: afd12b76bb524d0a907f0121ca68ae42, State: Running, Role: LEADER
I20260928 14:42:02.644560 10819 sys_catalog.cc:564] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:02.644594 10824 consensus_queue.cc:237] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afd12b76bb524d0a907f0121ca68ae42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } }
I20260928 14:42:02.645027 10825 sys_catalog.cc:455] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afd12b76bb524d0a907f0121ca68ae42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afd12b76bb524d0a907f0121ca68ae42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } } }
I20260928 14:42:02.645121 10825 sys_catalog.cc:458] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:02.645042 10826 sys_catalog.cc:455] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afd12b76bb524d0a907f0121ca68ae42. Latest consensus state: current_term: 1 leader_uuid: "afd12b76bb524d0a907f0121ca68ae42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afd12b76bb524d0a907f0121ca68ae42" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } } }
I20260928 14:42:02.645162 10826 sys_catalog.cc:458] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:02.646080 10831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:02.646656 10831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:02.648111 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:02.641735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58427 (local address 127.0.0.1:38131)
0928 14:42:02.641907 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:02.641915 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:02.641987 (+ 72us) server_negotiation.cc:407] Connection header received
0928 14:42:02.642527 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:02.642534 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:02.642592 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:02.642716 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:02.643046 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:02.644481 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:02.645654 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:02.646319 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:02.646470 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:02.647278 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:02.647282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:02.647282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:02.647499 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:02.647538 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:02.647553 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:02.647790 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:02.647823 (+ 33us) server_negotiation.cc:299] Negotiation successful
0928 14:42:02.647890 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260928 14:42:02.648145 10831 catalog_manager.cc:1347] Generated new cluster ID: 45e3734026ed41b18567e6ce5c997d46
I20260928 14:42:02.648288 10831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:42:02.649266 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:02.654207 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:02.650722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46601 (local address 127.0.0.1:38131)
0928 14:42:02.650758 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:02.650760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:02.650856 (+ 96us) server_negotiation.cc:407] Connection header received
0928 14:42:02.650901 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:02.650902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:02.650912 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:02.650985 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:02.651088 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:02.651811 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:02.652778 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:02.653143 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:02.653239 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:02.653961 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:02.653962 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:02.653962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:02.653967 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:02.653983 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:02.653988 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:02.654048 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:02.654104 (+ 56us) server_negotiation.cc:299] Negotiation successful
0928 14:42:02.654129 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:42:02.668989 10831 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:02.669776 10831 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:02.676261 10831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afd12b76bb524d0a907f0121ca68ae42: Generated new TSK 0
I20260928 14:42:02.676491 10831 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:02.679741 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 10751
2026-09-28T14:42:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1329 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:02 dist-test-slave-wwwb krb5kdc[10850](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:02 dist-test-slave-wwwb krb5kdc[10850](info): set up 2 sockets
Sep 28 14:42:02 dist-test-slave-wwwb krb5kdc[10850](info): commencing operation
krb5kdc: starting...
W20260928 14:42:03.698390 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:03 dist-test-slave-wwwb krb5kdc[10850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:42:03.815049 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41653
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:03.895705 10863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:03.895927 10863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:03.895951 10863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:03.897122 10863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:03.897158 10863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:03.897172 10863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:03.897185 10863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:03.897197 10863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:03.898747 10863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:03.898957 10863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:03.899123 10863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:03.903883 10869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:03.903911 10868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:03.903884 10871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:03.904098 10863 server_base.cc:1034] running on GCE node
I20260928 14:42:03.904340 10863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:03.904644 10863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:03.905793 10863 hybrid_clock.cc:648] HybridClock initialized: now 1790606523905776 us; error 36 us; skew 500 ppm
Sep 28 14:42:03 dist-test-slave-wwwb krb5kdc[10850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606523, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:03.908350 10863 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:42:03.908866 10863 webserver.cc:460] Webserver started at http://127.0.0.1:37695/ using document root <none> and password file <none>
I20260928 14:42:03.909114 10863 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:03.909185 10863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:03.909346 10863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:03.910516 10863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "b93239c162a64fc68f4719119349000d"
format_stamp: "Formatted at 2026-09-28 14:42:03 on dist-test-slave-wwwb"
server_key: "a2d9e2754fc8df69491389b38b6a4422"
server_key_iv: "95c267be8c3a4e235661235f1431746c"
server_key_version: "encryptionkey@0"
I20260928 14:42:03.910898 10863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "b93239c162a64fc68f4719119349000d"
format_stamp: "Formatted at 2026-09-28 14:42:03 on dist-test-slave-wwwb"
server_key: "a2d9e2754fc8df69491389b38b6a4422"
server_key_iv: "95c267be8c3a4e235661235f1431746c"
server_key_version: "encryptionkey@0"
I20260928 14:42:03.912287 10863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:42:03.913005 10878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:03.913184 10863 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:42:03.913282 10863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "b93239c162a64fc68f4719119349000d"
format_stamp: "Formatted at 2026-09-28 14:42:03 on dist-test-slave-wwwb"
server_key: "a2d9e2754fc8df69491389b38b6a4422"
server_key_iv: "95c267be8c3a4e235661235f1431746c"
server_key_version: "encryptionkey@0"
I20260928 14:42:03.913367 10863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:03.916896 10863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:03.919296 10863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:03.919459 10863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:03.923864 10863 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41653
I20260928 14:42:03.923900 10930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41653 every 8 connection(s)
I20260928 14:42:03.924258 10863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:03.925277 10931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:03.927750 10931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d: Bootstrap starting.
I20260928 14:42:03.928346 10931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:03.928632 10931 log.cc:824] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:03.929344 10931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d: No bootstrap required, opened a new log
I20260928 14:42:03.930426 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 10863
I20260928 14:42:03.930552 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:03.930763 22581 external_mini_cluster.cc:1428] Setting key 88f3c85f65e2f5436339a399a1406e08
I20260928 14:42:03.931526 10931 raft_consensus.cc:348] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b93239c162a64fc68f4719119349000d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41653 } }
I20260928 14:42:03.931670 10931 raft_consensus.cc:374] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:03.931692 10931 raft_consensus.cc:729] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b93239c162a64fc68f4719119349000d, State: Initialized, Role: FOLLOWER
I20260928 14:42:03.931838 10931 consensus_queue.cc:260] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b93239c162a64fc68f4719119349000d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41653 } }
I20260928 14:42:03.931901 10931 raft_consensus.cc:388] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:03.932003 10931 raft_consensus.cc:482] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:03.932085 10931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:03.932789 10931 raft_consensus.cc:504] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b93239c162a64fc68f4719119349000d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41653 } }
I20260928 14:42:03.932914 10931 leader_election.cc:302] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b93239c162a64fc68f4719119349000d; no voters:
I20260928 14:42:03.933127 10931 leader_election.cc:288] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:03.933148 10936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:03.933270 10936 raft_consensus.cc:686] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [term 1 LEADER]: Becoming Leader. State: Replica: b93239c162a64fc68f4719119349000d, State: Running, Role: LEADER
I20260928 14:42:03.933398 10935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:03.931361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34871 (local address 127.0.0.1:41653)
0928 14:42:03.931574 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:03.931587 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:03.931621 (+ 34us) server_negotiation.cc:407] Connection header received
0928 14:42:03.931752 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:03.931759 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:03.931790 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:03.931946 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:03.932792 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:03.932796 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:03.932796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:03.932800 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:03.932853 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:03.932865 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:03.932944 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:03.933012 (+ 68us) server_negotiation.cc:299] Negotiation successful
0928 14:42:03.933142 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
I20260928 14:42:03.933390 10936 consensus_queue.cc:237] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b93239c162a64fc68f4719119349000d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41653 } }
I20260928 14:42:03.933427 10931 sys_catalog.cc:564] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:03.934006 10938 sys_catalog.cc:455] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b93239c162a64fc68f4719119349000d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b93239c162a64fc68f4719119349000d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41653 } } }
I20260928 14:42:03.934008 10937 sys_catalog.cc:455] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b93239c162a64fc68f4719119349000d. Latest consensus state: current_term: 1 leader_uuid: "b93239c162a64fc68f4719119349000d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b93239c162a64fc68f4719119349000d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41653 } } }
I20260928 14:42:03.934103 10938 sys_catalog.cc:458] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:03.934105 10937 sys_catalog.cc:458] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:03.934644 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:03.934832 10942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:03.935343 10942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:03.937050 10942 catalog_manager.cc:1347] Generated new cluster ID: 0e9be23761b0492a905bb6451a24fbbb
I20260928 14:42:03.937114 10942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:42:03.937568 10935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:03.936229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57901 (local address 127.0.0.1:41653)
0928 14:42:03.936292 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:03.936294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:03.936366 (+ 72us) server_negotiation.cc:407] Connection header received
0928 14:42:03.936460 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:03.936462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:03.936467 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:03.936535 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:03.937335 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:03.937336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:03.937337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:03.937341 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:03.937357 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:03.937362 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:03.937432 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:03.937494 (+ 62us) server_negotiation.cc:299] Negotiation successful
0928 14:42:03.937510 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 14:42:03.947482 10942 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:03.947677 10942 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:03.953863 10942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b93239c162a64fc68f4719119349000d: Generated new TSK 0
I20260928 14:42:03.954159 10942 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:03.956733 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 10863
2026-09-28T14:42:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1276 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:04 dist-test-slave-wwwb krb5kdc[10962](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:04 dist-test-slave-wwwb krb5kdc[10962](info): set up 2 sockets
Sep 28 14:42:04 dist-test-slave-wwwb krb5kdc[10962](info): commencing operation
krb5kdc: starting...
W20260928 14:42:04.978518 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:05 dist-test-slave-wwwb krb5kdc[10962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:42:05.097229 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40097
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:05.179230 10975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:05.179490 10975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:05.179549 10975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:05.180781 10975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:05.180861 10975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:05.180886 10975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:05.180920 10975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:05.180969 10975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:05.182610 10975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:05.182937 10975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:05.183182 10975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:05.188186 10983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:05.188154 10980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:05.188216 10981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:05.188321 10975 server_base.cc:1034] running on GCE node
I20260928 14:42:05.188648 10975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:05.188951 10975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:05.190115 10975 hybrid_clock.cc:648] HybridClock initialized: now 1790606525190108 us; error 38 us; skew 500 ppm
Sep 28 14:42:05 dist-test-slave-wwwb krb5kdc[10962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606525, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:05.192641 10975 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:42:05.193081 10975 webserver.cc:460] Webserver started at http://127.0.0.1:39571/ using document root <none> and password file <none>
I20260928 14:42:05.193341 10975 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:05.193416 10975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:05.193562 10975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:05.194716 10975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "8a2ac3807e4f4b17b6c492e3b5fa9266"
format_stamp: "Formatted at 2026-09-28 14:42:05 on dist-test-slave-wwwb"
server_key: "94c9e5093419ad5f4cd272be09ad4303"
server_key_iv: "3604d66c610500d98e4b5d8fc9e852de"
server_key_version: "encryptionkey@0"
I20260928 14:42:05.195084 10975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "8a2ac3807e4f4b17b6c492e3b5fa9266"
format_stamp: "Formatted at 2026-09-28 14:42:05 on dist-test-slave-wwwb"
server_key: "94c9e5093419ad5f4cd272be09ad4303"
server_key_iv: "3604d66c610500d98e4b5d8fc9e852de"
server_key_version: "encryptionkey@0"
I20260928 14:42:05.196477 10975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:42:05.197214 10990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:05.197386 10975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:42:05.197479 10975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "8a2ac3807e4f4b17b6c492e3b5fa9266"
format_stamp: "Formatted at 2026-09-28 14:42:05 on dist-test-slave-wwwb"
server_key: "94c9e5093419ad5f4cd272be09ad4303"
server_key_iv: "3604d66c610500d98e4b5d8fc9e852de"
server_key_version: "encryptionkey@0"
I20260928 14:42:05.197562 10975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:05.205502 10975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:05.207510 10975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:05.207679 10975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:05.211786 10975 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40097
I20260928 14:42:05.211836 11042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40097 every 8 connection(s)
I20260928 14:42:05.212208 10975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:05.213091 11043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:05.215530 11043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266: Bootstrap starting.
I20260928 14:42:05.216167 11043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:05.216447 11043 log.cc:824] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:05.217154 11043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266: No bootstrap required, opened a new log
I20260928 14:42:05.218868 11043 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2ac3807e4f4b17b6c492e3b5fa9266" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40097 } }
I20260928 14:42:05.219019 11043 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:05.219043 11043 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a2ac3807e4f4b17b6c492e3b5fa9266, State: Initialized, Role: FOLLOWER
I20260928 14:42:05.219275 11043 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2ac3807e4f4b17b6c492e3b5fa9266" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40097 } }
I20260928 14:42:05.219362 11043 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:05.219413 11043 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:05.219465 11043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:05.220052 11043 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2ac3807e4f4b17b6c492e3b5fa9266" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40097 } }
I20260928 14:42:05.220182 11043 leader_election.cc:302] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a2ac3807e4f4b17b6c492e3b5fa9266; no voters:
I20260928 14:42:05.220376 11043 leader_election.cc:288] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:05.220423 11046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:05.220649 11043 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:05.220649 11046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [term 1 LEADER]: Becoming Leader. State: Replica: 8a2ac3807e4f4b17b6c492e3b5fa9266, State: Running, Role: LEADER
I20260928 14:42:05.220846 11046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2ac3807e4f4b17b6c492e3b5fa9266" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40097 } }
I20260928 14:42:05.221422 11047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a2ac3807e4f4b17b6c492e3b5fa9266" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2ac3807e4f4b17b6c492e3b5fa9266" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40097 } } }
I20260928 14:42:05.221441 11048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a2ac3807e4f4b17b6c492e3b5fa9266. Latest consensus state: current_term: 1 leader_uuid: "8a2ac3807e4f4b17b6c492e3b5fa9266" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2ac3807e4f4b17b6c492e3b5fa9266" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40097 } } }
I20260928 14:42:05.221527 11047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:05.221539 11048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:05.221755 11055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:05.222244 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 10975
I20260928 14:42:05.222379 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:05.222381 11055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:05.222592 22581 external_mini_cluster.cc:1428] Setting key bee3cf231e33877566f8589423876929
I20260928 14:42:05.224797 11055 catalog_manager.cc:1347] Generated new cluster ID: 9bdfe241aa6a4490bac6bbbdc0906d30
I20260928 14:42:05.224853 11055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:42:05 dist-test-slave-wwwb krb5kdc[10962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 14:42:05.235146 11065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:05.223086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:60751 (local address 127.0.0.1:40097)
0928 14:42:05.223234 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:05.223242 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:05.223376 (+ 134us) server_negotiation.cc:407] Connection header received
0928 14:42:05.223520 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:05.223528 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:05.223616 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:05.223773 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:05.224430 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:05.226001 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:05.227156 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:05.227816 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:05.227924 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:05.231116 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:05.231121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:05.231121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:05.231136 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:05.233158 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:05.233670 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:05.233673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:05.233674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:05.233789 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:05.234043 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:05.234044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:05.234044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:05.234361 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:05.234527 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:05.234675 (+ 148us) server_negotiation.cc:299] Negotiation successful
0928 14:42:05.234883 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":72,"threads_started":1}
I20260928 14:42:05.236187 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:05.239557 11055 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:05.240258 11055 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:05.245407 11065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:05.237705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:45839 (local address 127.0.0.1:40097)
0928 14:42:05.237806 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:05.237808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:05.237922 (+ 114us) server_negotiation.cc:407] Connection header received
0928 14:42:05.237995 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:05.237996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:05.238007 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:05.238077 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:05.238726 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:05.239521 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:05.240532 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:05.240873 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:05.240941 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:05.242397 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:05.242398 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:05.242398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:05.242406 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:05.244073 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:05.244534 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:05.244536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:05.244537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:05.244622 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:05.244854 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:05.244855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:05.244855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:05.244997 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:05.245123 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:05.245243 (+ 120us) server_negotiation.cc:299] Negotiation successful
0928 14:42:05.245312 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 14:42:05.253671 11055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a2ac3807e4f4b17b6c492e3b5fa9266: Generated new TSK 0
I20260928 14:42:05.253914 11055 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:05.257145 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 10975
W20260928 14:42:05.261590 10960 connection.cc:535] client connection to 127.0.0.1:40097 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40097): Connection reset by peer (error 104)
2026-09-28T14:42:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:05 dist-test-slave-wwwb krb5kdc[11074](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:05 dist-test-slave-wwwb krb5kdc[11074](info): set up 2 sockets
Sep 28 14:42:05 dist-test-slave-wwwb krb5kdc[11074](info): commencing operation
krb5kdc: starting...
W20260928 14:42:06.282191 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:06 dist-test-slave-wwwb krb5kdc[11074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:42:06.404364 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46265
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:06.487936 11087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:06.488168 11087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:06.488193 11087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:06.489596 11087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:06.489660 11087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:06.489677 11087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:06.489691 11087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:06.489702 11087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:06.491381 11087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:06.491600 11087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:06.491781 11087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:06.496593 11092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:06.496577 11093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:06.496577 11095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:06.496697 11087 server_base.cc:1034] running on GCE node
I20260928 14:42:06.497051 11087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:06.497344 11087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:06.498497 11087 hybrid_clock.cc:648] HybridClock initialized: now 1790606526498484 us; error 27 us; skew 500 ppm
Sep 28 14:42:06 dist-test-slave-wwwb krb5kdc[11074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606526, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:06.501041 11087 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:42:06.501544 11087 webserver.cc:460] Webserver started at http://127.0.0.1:44977/ using document root <none> and password file <none>
I20260928 14:42:06.501807 11087 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:06.501879 11087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:06.502051 11087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:06.503157 11087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "7d30a6e9f73b4535b53dcd10c634e9f5"
format_stamp: "Formatted at 2026-09-28 14:42:06 on dist-test-slave-wwwb"
server_key: "c55925eadbca0f57ef99081ce80b00b3"
server_key_iv: "3a0dff89d9833330e4280af2ba53c375"
server_key_version: "encryptionkey@0"
I20260928 14:42:06.503577 11087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "7d30a6e9f73b4535b53dcd10c634e9f5"
format_stamp: "Formatted at 2026-09-28 14:42:06 on dist-test-slave-wwwb"
server_key: "c55925eadbca0f57ef99081ce80b00b3"
server_key_iv: "3a0dff89d9833330e4280af2ba53c375"
server_key_version: "encryptionkey@0"
I20260928 14:42:06.504966 11087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:42:06.505725 11102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:06.505900 11087 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:42:06.506034 11087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "7d30a6e9f73b4535b53dcd10c634e9f5"
format_stamp: "Formatted at 2026-09-28 14:42:06 on dist-test-slave-wwwb"
server_key: "c55925eadbca0f57ef99081ce80b00b3"
server_key_iv: "3a0dff89d9833330e4280af2ba53c375"
server_key_version: "encryptionkey@0"
I20260928 14:42:06.506120 11087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:06.518729 11087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:06.520696 11087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:06.520875 11087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:06.525246 11087 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46265
I20260928 14:42:06.525281 11154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46265 every 8 connection(s)
I20260928 14:42:06.525655 11087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:06.526577 11155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:06.528767 11155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5: Bootstrap starting.
I20260928 14:42:06.529291 11155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:06.529521 11155 log.cc:824] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:06.529815 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 11087
I20260928 14:42:06.529937 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:06.530164 22581 external_mini_cluster.cc:1428] Setting key ef730fc0f1e0257dc5b32236c2212a99
I20260928 14:42:06.530437 11155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5: No bootstrap required, opened a new log
I20260928 14:42:06.532234 11155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d30a6e9f73b4535b53dcd10c634e9f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46265 } }
I20260928 14:42:06.532402 11155 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:06.532466 11155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d30a6e9f73b4535b53dcd10c634e9f5, State: Initialized, Role: FOLLOWER
I20260928 14:42:06.532606 11155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d30a6e9f73b4535b53dcd10c634e9f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46265 } }
I20260928 14:42:06.532709 11155 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:06.532755 11155 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:06.532805 11155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:06.533401 11155 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d30a6e9f73b4535b53dcd10c634e9f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46265 } }
I20260928 14:42:06.533525 11155 leader_election.cc:302] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d30a6e9f73b4535b53dcd10c634e9f5; no voters:
I20260928 14:42:06.533711 11155 leader_election.cc:288] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:06.533740 11160 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:06.533919 11160 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [term 1 LEADER]: Becoming Leader. State: Replica: 7d30a6e9f73b4535b53dcd10c634e9f5, State: Running, Role: LEADER
I20260928 14:42:06.534001 11155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:06.534050 11160 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d30a6e9f73b4535b53dcd10c634e9f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46265 } }
I20260928 14:42:06.535458 11161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d30a6e9f73b4535b53dcd10c634e9f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d30a6e9f73b4535b53dcd10c634e9f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46265 } } }
I20260928 14:42:06.535605 11161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [sys.catalog]: This master's current role is: LEADER
W20260928 14:42:06.535881 11174 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:42:06.535924 11174 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:42:06.535863 11162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d30a6e9f73b4535b53dcd10c634e9f5. Latest consensus state: current_term: 1 leader_uuid: "7d30a6e9f73b4535b53dcd10c634e9f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d30a6e9f73b4535b53dcd10c634e9f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46265 } } }
I20260928 14:42:06.536020 11162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:06.536269 11176 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:06.536458 11176 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:06.537416 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:06.530726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:38485 (local address 127.0.0.1:46265)
0928 14:42:06.530927 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:06.530936 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:06.530972 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:42:06.532214 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:06.532220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:06.532275 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:06.532422 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:06.532645 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:06.534150 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:06.534984 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:06.535623 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:06.535731 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:06.536742 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:06.536745 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:06.536746 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:06.536912 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:06.536945 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:06.536956 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:06.537029 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:06.537153 (+ 124us) server_negotiation.cc:299] Negotiation successful
0928 14:42:06.537202 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":83,"threads_started":1}
I20260928 14:42:06.538331 11176 catalog_manager.cc:1347] Generated new cluster ID: 340444b83e934473ba76a03afd6a3e96
I20260928 14:42:06.538389 11176 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:42:06.538419 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:06.543810 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:06.539986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:39343 (local address 127.0.0.1:46265)
0928 14:42:06.540026 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:06.540028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:06.540230 (+ 202us) server_negotiation.cc:407] Connection header received
0928 14:42:06.540294 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:06.540294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:06.540304 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:06.540362 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:06.540499 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:06.541254 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:06.542385 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:06.542713 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:06.542772 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:06.543537 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:06.543538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:06.543538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:06.543543 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:06.543557 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:06.543561 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:06.543609 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:06.543715 (+ 106us) server_negotiation.cc:299] Negotiation successful
0928 14:42:06.543729 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:42:06.546020 11176 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:06.546583 11176 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:06.552807 11176 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d30a6e9f73b4535b53dcd10c634e9f5: Generated new TSK 0
I20260928 14:42:06.553006 11176 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:06.558071 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 11087
W20260928 14:42:06.562485 11072 connection.cc:535] client connection to 127.0.0.1:46265 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46265): Connection reset by peer (error 104)
2026-09-28T14:42:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:06 dist-test-slave-wwwb krb5kdc[11186](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:06 dist-test-slave-wwwb krb5kdc[11186](info): set up 2 sockets
Sep 28 14:42:06 dist-test-slave-wwwb krb5kdc[11186](info): commencing operation
krb5kdc: starting...
W20260928 14:42:07.618438 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:07 dist-test-slave-wwwb krb5kdc[11186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:42:07.737365 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37673
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:07.820029 11199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:07.820255 11199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:07.820281 11199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:07.821434 11199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:07.821476 11199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:07.821491 11199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:07.821503 11199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:07.821516 11199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:07.823092 11199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:07.823312 11199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:07.823503 11199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:07.828666 11205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:07.828716 11199 server_base.cc:1034] running on GCE node
W20260928 14:42:07.828680 11207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:07.828692 11204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:07.829128 11199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:07.829453 11199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:07.830610 11199 hybrid_clock.cc:648] HybridClock initialized: now 1790606527830604 us; error 43 us; skew 500 ppm
Sep 28 14:42:07 dist-test-slave-wwwb krb5kdc[11186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:07.839223 11199 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:42:07.839772 11199 webserver.cc:460] Webserver started at http://127.0.0.1:39105/ using document root <none> and password file <none>
I20260928 14:42:07.840021 11199 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:07.840071 11199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:07.840253 11199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:07.841403 11199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "618473e16c0d4f7fb828b2841049ae56"
format_stamp: "Formatted at 2026-09-28 14:42:07 on dist-test-slave-wwwb"
server_key: "cf2f9aa2cbaca4b9bf5dabdc20384567"
server_key_iv: "0a46bbcc3f37015e641db5caa13f6cd6"
server_key_version: "encryptionkey@0"
I20260928 14:42:07.841791 11199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "618473e16c0d4f7fb828b2841049ae56"
format_stamp: "Formatted at 2026-09-28 14:42:07 on dist-test-slave-wwwb"
server_key: "cf2f9aa2cbaca4b9bf5dabdc20384567"
server_key_iv: "0a46bbcc3f37015e641db5caa13f6cd6"
server_key_version: "encryptionkey@0"
I20260928 14:42:07.843281 11199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:07.844105 11214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:07.844323 11199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:07.844426 11199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "618473e16c0d4f7fb828b2841049ae56"
format_stamp: "Formatted at 2026-09-28 14:42:07 on dist-test-slave-wwwb"
server_key: "cf2f9aa2cbaca4b9bf5dabdc20384567"
server_key_iv: "0a46bbcc3f37015e641db5caa13f6cd6"
server_key_version: "encryptionkey@0"
I20260928 14:42:07.844515 11199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:07.848333 11199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:07.850951 11199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:07.851125 11199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:07.855703 11199 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37673
I20260928 14:42:07.855782 11266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37673 every 8 connection(s)
I20260928 14:42:07.856112 11199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:07.857105 11267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:07.859539 11267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56: Bootstrap starting.
I20260928 14:42:07.860140 11267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:07.860440 11267 log.cc:824] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:07.861164 11267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56: No bootstrap required, opened a new log
I20260928 14:42:07.862394 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 11199
I20260928 14:42:07.862540 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:07.862723 22581 external_mini_cluster.cc:1428] Setting key e505b088e1868e93957781f60a126f4d
I20260928 14:42:07.863119 11267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "618473e16c0d4f7fb828b2841049ae56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37673 } }
I20260928 14:42:07.863328 11267 raft_consensus.cc:374] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:07.863408 11267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 618473e16c0d4f7fb828b2841049ae56, State: Initialized, Role: FOLLOWER
I20260928 14:42:07.863606 11267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "618473e16c0d4f7fb828b2841049ae56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37673 } }
I20260928 14:42:07.863698 11267 raft_consensus.cc:388] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:07.863730 11267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:07.863770 11267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:07.864635 11267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "618473e16c0d4f7fb828b2841049ae56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37673 } }
I20260928 14:42:07.864751 11267 leader_election.cc:302] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 618473e16c0d4f7fb828b2841049ae56; no voters:
I20260928 14:42:07.864923 11267 leader_election.cc:288] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:07.864956 11272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:07.865097 11272 raft_consensus.cc:686] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [term 1 LEADER]: Becoming Leader. State: Replica: 618473e16c0d4f7fb828b2841049ae56, State: Running, Role: LEADER
I20260928 14:42:07.865207 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:07.863321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:49895 (local address 127.0.0.1:37673)
0928 14:42:07.863513 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:07.863523 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:07.863556 (+ 33us) server_negotiation.cc:407] Connection header received
0928 14:42:07.863684 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:07.863692 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:07.863725 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:07.863882 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:07.864566 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:07.864570 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:07.864570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:07.864575 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:07.864622 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:07.864638 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:07.864727 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:07.864901 (+ 174us) server_negotiation.cc:299] Negotiation successful
0928 14:42:07.864955 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260928 14:42:07.865237 11272 consensus_queue.cc:237] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "618473e16c0d4f7fb828b2841049ae56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37673 } }
I20260928 14:42:07.865217 11267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:07.865650 11274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 618473e16c0d4f7fb828b2841049ae56. Latest consensus state: current_term: 1 leader_uuid: "618473e16c0d4f7fb828b2841049ae56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "618473e16c0d4f7fb828b2841049ae56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37673 } } }
I20260928 14:42:07.865759 11274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:07.865666 11273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "618473e16c0d4f7fb828b2841049ae56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "618473e16c0d4f7fb828b2841049ae56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37673 } } }
I20260928 14:42:07.866031 11273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:07.866215 11276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:07.866315 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:07.866793 11276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:07.868580 11276 catalog_manager.cc:1347] Generated new cluster ID: d948553ff2cc490eb6c4731959ce0c87
I20260928 14:42:07.868635 11276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:42:07.869316 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:07.867910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:53493 (local address 127.0.0.1:37673)
0928 14:42:07.867946 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:07.867948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:07.868179 (+ 231us) server_negotiation.cc:407] Connection header received
0928 14:42:07.868260 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:07.868261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:07.868265 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:07.868339 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:07.869076 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:07.869078 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:07.869078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:07.869082 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:07.869099 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:07.869103 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:07.869164 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:07.869223 (+ 59us) server_negotiation.cc:299] Negotiation successful
0928 14:42:07.869238 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:42:07.876376 11276 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:07.876574 11276 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:07.880632 11276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 618473e16c0d4f7fb828b2841049ae56: Generated new TSK 0
I20260928 14:42:07.880858 11276 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:07.883527 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 11199
2026-09-28T14:42:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:07 dist-test-slave-wwwb krb5kdc[11298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:07 dist-test-slave-wwwb krb5kdc[11298](info): set up 2 sockets
Sep 28 14:42:07 dist-test-slave-wwwb krb5kdc[11298](info): commencing operation
krb5kdc: starting...
W20260928 14:42:08.910439 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:08 dist-test-slave-wwwb krb5kdc[11298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:42:09.034888 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36931
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:09.117915 11311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:09.118196 11311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:09.118222 11311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:09.119589 11311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:09.119637 11311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:09.119652 11311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:09.119665 11311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:09.119678 11311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:09.121282 11311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36931
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:09.121570 11311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:09.121815 11311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:09.126976 11317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:09.126940 11319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:09.127121 11316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:09.127175 11311 server_base.cc:1034] running on GCE node
I20260928 14:42:09.127422 11311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:09.127704 11311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:09.128854 11311 hybrid_clock.cc:648] HybridClock initialized: now 1790606529128840 us; error 31 us; skew 500 ppm
Sep 28 14:42:09 dist-test-slave-wwwb krb5kdc[11298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606529, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:09.131635 11311 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:42:09.132128 11311 webserver.cc:460] Webserver started at http://127.0.0.1:33005/ using document root <none> and password file <none>
I20260928 14:42:09.132380 11311 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:09.132452 11311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:09.132603 11311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:09.133785 11311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "b219fa645c6548c39c9d5b0545966a92"
format_stamp: "Formatted at 2026-09-28 14:42:09 on dist-test-slave-wwwb"
server_key: "41775de15d62ca29c39a129f1f90b06f"
server_key_iv: "f4aab64763841b3ea1a75397cbc22003"
server_key_version: "encryptionkey@0"
I20260928 14:42:09.134238 11311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "b219fa645c6548c39c9d5b0545966a92"
format_stamp: "Formatted at 2026-09-28 14:42:09 on dist-test-slave-wwwb"
server_key: "41775de15d62ca29c39a129f1f90b06f"
server_key_iv: "f4aab64763841b3ea1a75397cbc22003"
server_key_version: "encryptionkey@0"
I20260928 14:42:09.135695 11311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:09.136497 11326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:09.136683 11311 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:42:09.136801 11311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "b219fa645c6548c39c9d5b0545966a92"
format_stamp: "Formatted at 2026-09-28 14:42:09 on dist-test-slave-wwwb"
server_key: "41775de15d62ca29c39a129f1f90b06f"
server_key_iv: "f4aab64763841b3ea1a75397cbc22003"
server_key_version: "encryptionkey@0"
I20260928 14:42:09.136889 11311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:09.147605 11311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:09.149569 11311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:09.149758 11311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:09.153789 11311 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36931
I20260928 14:42:09.153831 11378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36931 every 8 connection(s)
I20260928 14:42:09.154263 11311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:09.155208 11379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:09.157639 11379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92: Bootstrap starting.
I20260928 14:42:09.158344 11379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:09.158665 11379 log.cc:824] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:09.159474 11379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92: No bootstrap required, opened a new log
I20260928 14:42:09.160233 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 11311
I20260928 14:42:09.160343 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:09.160559 22581 external_mini_cluster.cc:1428] Setting key 6b5d77cb7748e003e9b038b535ba9a45
I20260928 14:42:09.161825 11379 raft_consensus.cc:348] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b219fa645c6548c39c9d5b0545966a92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36931 } }
I20260928 14:42:09.162053 11379 raft_consensus.cc:374] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:09.162098 11379 raft_consensus.cc:729] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b219fa645c6548c39c9d5b0545966a92, State: Initialized, Role: FOLLOWER
I20260928 14:42:09.162262 11379 consensus_queue.cc:260] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b219fa645c6548c39c9d5b0545966a92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36931 } }
I20260928 14:42:09.162331 11379 raft_consensus.cc:388] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:09.162355 11379 raft_consensus.cc:482] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:09.162433 11379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:09.163676 11379 raft_consensus.cc:504] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b219fa645c6548c39c9d5b0545966a92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36931 } }
I20260928 14:42:09.163793 11379 leader_election.cc:302] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b219fa645c6548c39c9d5b0545966a92; no voters:
I20260928 14:42:09.163955 11379 leader_election.cc:288] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:09.164037 11384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:09.164208 11379 sys_catalog.cc:564] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:09.164258 11384 raft_consensus.cc:686] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [term 1 LEADER]: Becoming Leader. State: Replica: b219fa645c6548c39c9d5b0545966a92, State: Running, Role: LEADER
I20260928 14:42:09.164371 11384 consensus_queue.cc:237] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b219fa645c6548c39c9d5b0545966a92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36931 } }
I20260928 14:42:09.164877 11385 sys_catalog.cc:455] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b219fa645c6548c39c9d5b0545966a92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b219fa645c6548c39c9d5b0545966a92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36931 } } }
I20260928 14:42:09.164958 11385 sys_catalog.cc:458] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:09.165114 11386 sys_catalog.cc:455] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b219fa645c6548c39c9d5b0545966a92. Latest consensus state: current_term: 1 leader_uuid: "b219fa645c6548c39c9d5b0545966a92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b219fa645c6548c39c9d5b0545966a92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36931 } } }
I20260928 14:42:09.165179 11386 sys_catalog.cc:458] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92 [sys.catalog]: This master's current role is: LEADER
W20260928 14:42:09.166208 11400 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:42:09.166316 11400 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:42:09.166406 11393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:09.166611 11393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 28 14:42:09 dist-test-slave-wwwb krb5kdc[11298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 14:42:09.169364 11393 catalog_manager.cc:1347] Generated new cluster ID: b00910475b1d411c88a8a4244bfd52dd
I20260928 14:42:09.169457 11393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:42:09.173002 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:09.161134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:57327 (local address 127.0.0.1:36931)
0928 14:42:09.161366 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:09.161377 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:09.161418 (+ 41us) server_negotiation.cc:407] Connection header received
0928 14:42:09.161769 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:09.161779 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:09.161853 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:09.162043 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:09.162863 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:09.164323 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:09.165552 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:09.166184 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:09.166312 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:09.169348 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:09.169352 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:09.169353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:09.169364 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:09.171257 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:09.171719 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:09.171723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:09.171723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:09.171844 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:09.172093 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:09.172094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:09.172095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:09.172371 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:09.172551 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:09.172672 (+ 121us) server_negotiation.cc:299] Negotiation successful
0928 14:42:09.172784 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":106,"threads_started":1}
I20260928 14:42:09.174002 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:09.188884 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:09.175369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:34077 (local address 127.0.0.1:36931)
0928 14:42:09.175410 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:09.175411 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:09.182232 (+ 6821us) server_negotiation.cc:407] Connection header received
0928 14:42:09.182335 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:09.182336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:09.182349 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:09.182430 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:09.183060 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:09.183613 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:09.184369 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:09.184605 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:09.184691 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:09.186074 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:09.186075 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:09.186076 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:09.186084 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:09.187580 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:09.188040 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:09.188042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:09.188042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:09.188104 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:09.188331 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:09.188332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:09.188332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:09.188452 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:09.188574 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:09.188708 (+ 134us) server_negotiation.cc:299] Negotiation successful
0928 14:42:09.188775 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:42:09.189267 11393 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:09.189917 11393 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:09.197467 11393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b219fa645c6548c39c9d5b0545966a92: Generated new TSK 0
I20260928 14:42:09.197721 11393 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:09.202893 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 11311
W20260928 14:42:09.206903 11296 connection.cc:535] client connection to 127.0.0.1:36931 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36931): Connection reset by peer (error 104)
2026-09-28T14:42:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1319 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:09 dist-test-slave-wwwb krb5kdc[11410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:09 dist-test-slave-wwwb krb5kdc[11410](info): set up 2 sockets
Sep 28 14:42:09 dist-test-slave-wwwb krb5kdc[11410](info): commencing operation
krb5kdc: starting...
W20260928 14:42:10.246411 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:10 dist-test-slave-wwwb krb5kdc[11410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:42:10.368608 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35405
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:10.457664 11423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:10.457912 11423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:10.457962 11423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:10.459168 11423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:10.459213 11423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:10.459227 11423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:10.459239 11423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:10.459251 11423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:10.460774 11423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35405
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:10.461001 11423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:10.461192 11423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:10.466411 11429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:10.466461 11423 server_base.cc:1034] running on GCE node
W20260928 14:42:10.466394 11428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:10.466398 11431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:10.466938 11423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:10.467286 11423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:10.468466 11423 hybrid_clock.cc:648] HybridClock initialized: now 1790606530468447 us; error 36 us; skew 500 ppm
Sep 28 14:42:10 dist-test-slave-wwwb krb5kdc[11410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606530, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:10.471326 11423 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:42:10.471824 11423 webserver.cc:460] Webserver started at http://127.0.0.1:36553/ using document root <none> and password file <none>
I20260928 14:42:10.472077 11423 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:10.472151 11423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:10.472316 11423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:10.473507 11423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "d852248f801347559f94fe111971fd24"
format_stamp: "Formatted at 2026-09-28 14:42:10 on dist-test-slave-wwwb"
server_key: "545ec5c3c0eb8fc0bad482ea505aecb2"
server_key_iv: "3bf474f4d9e7d2486f222742464a4cfc"
server_key_version: "encryptionkey@0"
I20260928 14:42:10.473919 11423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "d852248f801347559f94fe111971fd24"
format_stamp: "Formatted at 2026-09-28 14:42:10 on dist-test-slave-wwwb"
server_key: "545ec5c3c0eb8fc0bad482ea505aecb2"
server_key_iv: "3bf474f4d9e7d2486f222742464a4cfc"
server_key_version: "encryptionkey@0"
I20260928 14:42:10.475524 11423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:10.476464 11438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:10.476675 11423 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:10.476804 11423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "d852248f801347559f94fe111971fd24"
format_stamp: "Formatted at 2026-09-28 14:42:10 on dist-test-slave-wwwb"
server_key: "545ec5c3c0eb8fc0bad482ea505aecb2"
server_key_iv: "3bf474f4d9e7d2486f222742464a4cfc"
server_key_version: "encryptionkey@0"
I20260928 14:42:10.476889 11423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:10.502952 11423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:10.505473 11423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:10.505653 11423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:10.509639 11423 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35405
I20260928 14:42:10.509677 11490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35405 every 8 connection(s)
I20260928 14:42:10.510097 11423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:10.511051 11491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:10.513453 11491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24: Bootstrap starting.
I20260928 14:42:10.514129 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 11423
I20260928 14:42:10.514173 11491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:10.514271 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:10.514490 22581 external_mini_cluster.cc:1428] Setting key 7e74efe9eac1a5ea90fea8c07a70c698
I20260928 14:42:10.514528 11491 log.cc:824] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:10.515564 11491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24: No bootstrap required, opened a new log
I20260928 14:42:10.518303 11491 raft_consensus.cc:348] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d852248f801347559f94fe111971fd24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } }
I20260928 14:42:10.518466 11491 raft_consensus.cc:374] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:10.518491 11491 raft_consensus.cc:729] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d852248f801347559f94fe111971fd24, State: Initialized, Role: FOLLOWER
I20260928 14:42:10.518643 11491 consensus_queue.cc:260] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d852248f801347559f94fe111971fd24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } }
I20260928 14:42:10.518723 11491 raft_consensus.cc:388] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:10.518769 11491 raft_consensus.cc:482] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:10.518821 11491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:10.519642 11491 raft_consensus.cc:504] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d852248f801347559f94fe111971fd24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } }
I20260928 14:42:10.519767 11491 leader_election.cc:302] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d852248f801347559f94fe111971fd24; no voters:
I20260928 14:42:10.519971 11491 leader_election.cc:288] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:10.520035 11496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:10.520267 11496 raft_consensus.cc:686] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [term 1 LEADER]: Becoming Leader. State: Replica: d852248f801347559f94fe111971fd24, State: Running, Role: LEADER
I20260928 14:42:10.520383 11491 sys_catalog.cc:564] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:10.520395 11496 consensus_queue.cc:237] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d852248f801347559f94fe111971fd24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } }
I20260928 14:42:10.520880 11498 sys_catalog.cc:455] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d852248f801347559f94fe111971fd24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d852248f801347559f94fe111971fd24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } } }
I20260928 14:42:10.520862 11496 sys_catalog.cc:455] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d852248f801347559f94fe111971fd24. Latest consensus state: current_term: 1 leader_uuid: "d852248f801347559f94fe111971fd24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d852248f801347559f94fe111971fd24" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35405 } } }
I20260928 14:42:10.521070 11496 sys_catalog.cc:458] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:10.521064 11498 sys_catalog.cc:458] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:10.521405 11502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:10.522389 11495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:10.515056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:60801 (local address 127.0.0.1:35405)
0928 14:42:10.515247 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:10.515257 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:10.515292 (+ 35us) server_negotiation.cc:407] Connection header received
0928 14:42:10.515936 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:10.515946 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:10.516028 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:10.516203 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:10.516457 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:10.517978 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:10.519098 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:10.519751 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:10.519866 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:10.520798 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:10.520952 (+ 154us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:10.520954 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:10.521266 (+ 312us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:10.521315 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:10.521334 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:10.521701 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:10.521752 (+ 51us) server_negotiation.cc:299] Negotiation successful
0928 14:42:10.521782 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":104,"threads_started":1}
I20260928 14:42:10.522416 11502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:10.523874 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:10.524461 11502 catalog_manager.cc:1347] Generated new cluster ID: bbe0f89623f04c41a0a540c813c0bd87
I20260928 14:42:10.524503 11502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:42:10.529229 11495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:10.525548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:48337 (local address 127.0.0.1:35405)
0928 14:42:10.525588 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:10.525590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:10.525599 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:42:10.525665 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:10.525666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:10.525675 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:10.525745 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:10.525918 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:10.526680 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:10.527774 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:10.528090 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:10.528152 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:10.528943 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:10.528944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:10.528945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:10.528950 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:10.528963 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:10.528968 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:10.529023 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:10.529135 (+ 112us) server_negotiation.cc:299] Negotiation successful
0928 14:42:10.529146 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:42:10.542182 11502 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:10.542841 11502 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:10.549186 11502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d852248f801347559f94fe111971fd24: Generated new TSK 0
I20260928 14:42:10.549434 11502 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:10.557758 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 11423
W20260928 14:42:10.562196 11408 connection.cc:535] client connection to 127.0.0.1:35405 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35405): Connection reset by peer (error 104)
2026-09-28T14:42:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1355 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:10 dist-test-slave-wwwb krb5kdc[11522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:10 dist-test-slave-wwwb krb5kdc[11522](info): set up 2 sockets
Sep 28 14:42:10 dist-test-slave-wwwb krb5kdc[11522](info): commencing operation
krb5kdc: starting...
W20260928 14:42:11.590404 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:11 dist-test-slave-wwwb krb5kdc[11522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:42:11.716612 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38561
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:11.799471 11535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:11.799721 11535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:11.799747 11535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:11.801278 11535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:11.801340 11535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:11.801355 11535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:11.801369 11535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:11.801422 11535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:11.803421 11535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38561
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:11.803774 11535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:11.804046 11535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:11.809188 11543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:11.809186 11541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:11.809180 11540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:11.809222 11535 server_base.cc:1034] running on GCE node
I20260928 14:42:11.809628 11535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:11.810031 11535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:11.811219 11535 hybrid_clock.cc:648] HybridClock initialized: now 1790606531811192 us; error 39 us; skew 500 ppm
Sep 28 14:42:11 dist-test-slave-wwwb krb5kdc[11522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606531, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:11.813797 11535 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:42:11.814389 11535 webserver.cc:460] Webserver started at http://127.0.0.1:41027/ using document root <none> and password file <none>
I20260928 14:42:11.814657 11535 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:11.814733 11535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:11.814898 11535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:11.816044 11535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "afab7cffd34c4ff0b76b77bfbdf5f9de"
format_stamp: "Formatted at 2026-09-28 14:42:11 on dist-test-slave-wwwb"
server_key: "983f182853a35e95e52436ba24a33e0d"
server_key_iv: "954096fbcc2c59e4b1703edbba169211"
server_key_version: "encryptionkey@0"
I20260928 14:42:11.816440 11535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "afab7cffd34c4ff0b76b77bfbdf5f9de"
format_stamp: "Formatted at 2026-09-28 14:42:11 on dist-test-slave-wwwb"
server_key: "983f182853a35e95e52436ba24a33e0d"
server_key_iv: "954096fbcc2c59e4b1703edbba169211"
server_key_version: "encryptionkey@0"
I20260928 14:42:11.817909 11535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:42:11.818799 11550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:11.819121 11535 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:11.819235 11535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "afab7cffd34c4ff0b76b77bfbdf5f9de"
format_stamp: "Formatted at 2026-09-28 14:42:11 on dist-test-slave-wwwb"
server_key: "983f182853a35e95e52436ba24a33e0d"
server_key_iv: "954096fbcc2c59e4b1703edbba169211"
server_key_version: "encryptionkey@0"
I20260928 14:42:11.819316 11535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:11.822954 11535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:11.825181 11535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:11.825358 11535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:11.829725 11535 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38561
I20260928 14:42:11.829775 11602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38561 every 8 connection(s)
I20260928 14:42:11.830237 11535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:11.831297 11603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:11.831620 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 11535
I20260928 14:42:11.831760 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:11.831943 22581 external_mini_cluster.cc:1428] Setting key b2153202798974bfcf0e1c900e891427
I20260928 14:42:11.834528 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:11.832527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:60759 (local address 127.0.0.1:38561)
0928 14:42:11.832726 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:11.832734 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:11.832762 (+ 28us) server_negotiation.cc:407] Connection header received
0928 14:42:11.833188 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:11.833195 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:11.833221 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:11.833351 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:11.834064 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:11.834067 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:11.834068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:11.834072 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:11.834113 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:11.834124 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:11.834183 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:11.834271 (+ 88us) server_negotiation.cc:299] Negotiation successful
0928 14:42:11.834328 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260928 14:42:11.834591 11603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de: Bootstrap starting.
I20260928 14:42:11.835322 11603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:11.835619 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:11.835630 11603 log.cc:824] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:11.836346 11603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de: No bootstrap required, opened a new log
I20260928 14:42:11.838315 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:11.837108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:37997 (local address 127.0.0.1:38561)
0928 14:42:11.837150 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:11.837152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:11.837300 (+ 148us) server_negotiation.cc:407] Connection header received
0928 14:42:11.837380 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:11.837381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:11.837386 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:11.837444 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:11.838086 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:11.838087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:11.838088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:11.838093 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:11.838106 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:11.838111 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:11.838169 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:11.838232 (+ 63us) server_negotiation.cc:299] Negotiation successful
0928 14:42:11.838248 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:42:11.838668 11603 raft_consensus.cc:348] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afab7cffd34c4ff0b76b77bfbdf5f9de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38561 } }
I20260928 14:42:11.838845 11603 raft_consensus.cc:374] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:11.838882 11603 raft_consensus.cc:729] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afab7cffd34c4ff0b76b77bfbdf5f9de, State: Initialized, Role: FOLLOWER
I20260928 14:42:11.838991 11603 consensus_queue.cc:260] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afab7cffd34c4ff0b76b77bfbdf5f9de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38561 } }
I20260928 14:42:11.839057 11603 raft_consensus.cc:388] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:11.839080 11603 raft_consensus.cc:482] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:11.839107 11603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:11.839644 11603 raft_consensus.cc:504] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afab7cffd34c4ff0b76b77bfbdf5f9de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38561 } }
I20260928 14:42:11.839732 11603 leader_election.cc:302] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afab7cffd34c4ff0b76b77bfbdf5f9de; no voters:
I20260928 14:42:11.839852 11603 leader_election.cc:288] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:11.839936 11613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:11.840106 11613 raft_consensus.cc:686] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [term 1 LEADER]: Becoming Leader. State: Replica: afab7cffd34c4ff0b76b77bfbdf5f9de, State: Running, Role: LEADER
I20260928 14:42:11.840167 11603 sys_catalog.cc:564] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:11.840242 11613 consensus_queue.cc:237] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afab7cffd34c4ff0b76b77bfbdf5f9de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38561 } }
I20260928 14:42:11.840631 11613 sys_catalog.cc:455] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [sys.catalog]: SysCatalogTable state changed. Reason: New leader afab7cffd34c4ff0b76b77bfbdf5f9de. Latest consensus state: current_term: 1 leader_uuid: "afab7cffd34c4ff0b76b77bfbdf5f9de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afab7cffd34c4ff0b76b77bfbdf5f9de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38561 } } }
I20260928 14:42:11.840778 11613 sys_catalog.cc:458] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:11.840642 11615 sys_catalog.cc:455] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afab7cffd34c4ff0b76b77bfbdf5f9de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afab7cffd34c4ff0b76b77bfbdf5f9de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38561 } } }
I20260928 14:42:11.841019 11615 sys_catalog.cc:458] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de [sys.catalog]: This master's current role is: LEADER
W20260928 14:42:11.842060 11628 catalog_manager.cc:1558] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:42:11.842130 11628 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:42:11.842187 11629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:11.842347 11629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:11.843767 11629 catalog_manager.cc:1347] Generated new cluster ID: 48f5e083e51f4de4a6513175a4ece4ff
I20260928 14:42:11.843823 11629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:42:11.872653 11629 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:11.872879 11629 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:11.877981 11629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afab7cffd34c4ff0b76b77bfbdf5f9de: Generated new TSK 0
I20260928 14:42:11.878232 11629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:11.884914 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 11535
2026-09-28T14:42:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:11 dist-test-slave-wwwb krb5kdc[11634](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:11 dist-test-slave-wwwb krb5kdc[11634](info): set up 2 sockets
Sep 28 14:42:11 dist-test-slave-wwwb krb5kdc[11634](info): commencing operation
krb5kdc: starting...
W20260928 14:42:12.918435 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:13 dist-test-slave-wwwb krb5kdc[11634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:42:13.037165 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:40851
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:44575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:40851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:13.119452 11647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:13.119745 11647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:13.119788 11647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:13.121048 11647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:13.121136 11647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:13.121160 11647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:13.121181 11647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:13.121208 11647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:13.122849 11647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:44575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:40851
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:40851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:13.123178 11647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:13.123420 11647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:13.128430 11652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:13.128424 11653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:13.128424 11655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:13.128755 11647 server_base.cc:1034] running on GCE node
I20260928 14:42:13.129065 11647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:13.129420 11647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:13.130594 11647 hybrid_clock.cc:648] HybridClock initialized: now 1790606533130586 us; error 40 us; skew 500 ppm
Sep 28 14:42:13 dist-test-slave-wwwb krb5kdc[11634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606533, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:13.138473 11647 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:42:13.139063 11647 webserver.cc:460] Webserver started at http://127.22.13.126:33437/ using document root <none> and password file <none>
I20260928 14:42:13.139323 11647 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:13.139402 11647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:13.139571 11647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:13.140758 11647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "ea2e4de8c89e433fa912c1d84e2902a4"
format_stamp: "Formatted at 2026-09-28 14:42:13 on dist-test-slave-wwwb"
server_key: "2540c2e34378d3d7e67543a895845693"
server_key_iv: "a764763fcd29ee379cb5496e9f03ae3e"
server_key_version: "encryptionkey@0"
I20260928 14:42:13.141185 11647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "ea2e4de8c89e433fa912c1d84e2902a4"
format_stamp: "Formatted at 2026-09-28 14:42:13 on dist-test-slave-wwwb"
server_key: "2540c2e34378d3d7e67543a895845693"
server_key_iv: "a764763fcd29ee379cb5496e9f03ae3e"
server_key_version: "encryptionkey@0"
I20260928 14:42:13.142978 11647 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260928 14:42:13.143821 11662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:13.144021 11647 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:13.144138 11647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "ea2e4de8c89e433fa912c1d84e2902a4"
format_stamp: "Formatted at 2026-09-28 14:42:13 on dist-test-slave-wwwb"
server_key: "2540c2e34378d3d7e67543a895845693"
server_key_iv: "a764763fcd29ee379cb5496e9f03ae3e"
server_key_version: "encryptionkey@0"
I20260928 14:42:13.144238 11647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:13.178937 11647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:13.180984 11647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:13.181180 11647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:13.185832 11647 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:40851
I20260928 14:42:13.185870 11714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:40851 every 8 connection(s)
I20260928 14:42:13.186298 11647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:13.187278 11715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:13.189720 11715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4: Bootstrap starting.
I20260928 14:42:13.190376 11715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:13.190685 11715 log.cc:824] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:13.191465 11715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4: No bootstrap required, opened a new log
I20260928 14:42:13.192772 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 11647
I20260928 14:42:13.192916 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:13.193125 22581 external_mini_cluster.cc:1428] Setting key 0f6ae8c96952f9fdcc5f6982bfae7cb9
I20260928 14:42:13.193266 11715 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2e4de8c89e433fa912c1d84e2902a4" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40851 } }
I20260928 14:42:13.193482 11715 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:13.193533 11715 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea2e4de8c89e433fa912c1d84e2902a4, State: Initialized, Role: FOLLOWER
I20260928 14:42:13.193722 11715 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2e4de8c89e433fa912c1d84e2902a4" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40851 } }
I20260928 14:42:13.193833 11715 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:13.193897 11715 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:13.193997 11715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:13.195122 11715 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2e4de8c89e433fa912c1d84e2902a4" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40851 } }
I20260928 14:42:13.195235 11715 leader_election.cc:302] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea2e4de8c89e433fa912c1d84e2902a4; no voters:
I20260928 14:42:13.195389 11715 leader_election.cc:288] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:13.195485 11720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:13.195642 11715 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:13.195684 11720 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [term 1 LEADER]: Becoming Leader. State: Replica: ea2e4de8c89e433fa912c1d84e2902a4, State: Running, Role: LEADER
I20260928 14:42:13.195860 11720 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2e4de8c89e433fa912c1d84e2902a4" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40851 } }
I20260928 14:42:13.196255 11721 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea2e4de8c89e433fa912c1d84e2902a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2e4de8c89e433fa912c1d84e2902a4" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40851 } } }
I20260928 14:42:13.196378 11721 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:13.196261 11722 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea2e4de8c89e433fa912c1d84e2902a4. Latest consensus state: current_term: 1 leader_uuid: "ea2e4de8c89e433fa912c1d84e2902a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2e4de8c89e433fa912c1d84e2902a4" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 40851 } } }
I20260928 14:42:13.196429 11722 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:13.196614 11729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:13.197227 11729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 28 14:42:13 dist-test-slave-wwwb krb5kdc[11634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:42:13.201426 11729 catalog_manager.cc:1347] Generated new cluster ID: 9f4c5d8245e14f0aae4ff956f793257b
I20260928 14:42:13.201514 11729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:42:13.205821 11719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:13.193737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47743 (local address 127.22.13.126:40851)
0928 14:42:13.193952 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:13.193963 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:13.193999 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:42:13.194131 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:13.194138 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:13.194206 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:13.194351 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:13.195019 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:13.196428 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:13.197615 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:13.198256 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:13.198388 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:13.201390 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:13.201395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:13.201395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:13.201409 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:13.203709 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:13.204317 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:13.204320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:13.204320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:13.204459 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:13.204754 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:13.204755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:13.204755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:13.205104 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:13.205269 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:13.205449 (+ 180us) server_negotiation.cc:299] Negotiation successful
0928 14:42:13.205570 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":102,"threads_started":1}
I20260928 14:42:13.207042 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:13.210390 11729 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:13.210922 11729 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:13.216621 11719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:13.208730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48875 (local address 127.22.13.126:40851)
0928 14:42:13.208768 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:13.208770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:13.208799 (+ 29us) server_negotiation.cc:407] Connection header received
0928 14:42:13.208822 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:13.208823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:13.208842 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:13.208896 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:13.209622 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:13.210431 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:13.211520 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:13.211865 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:13.211906 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:13.213503 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:13.213504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:13.213505 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:13.213512 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:13.215228 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:13.215710 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:13.215711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:13.215711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:13.215776 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:13.216032 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:13.216032 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:13.216033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:13.216172 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:13.216276 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:13.216448 (+ 172us) server_negotiation.cc:299] Negotiation successful
0928 14:42:13.216517 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:42:13.220779 11729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea2e4de8c89e433fa912c1d84e2902a4: Generated new TSK 0
I20260928 14:42:13.221035 11729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:13.227582 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 11647
W20260928 14:42:13.231594 11632 connection.cc:535] client connection to 127.22.13.126:40851 recv error: Network error: failed to read from TLS socket (remote: 127.22.13.126:40851): Connection reset by peer (error 104)
2026-09-28T14:42:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:13 dist-test-slave-wwwb krb5kdc[11746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:13 dist-test-slave-wwwb krb5kdc[11746](info): set up 2 sockets
Sep 28 14:42:13 dist-test-slave-wwwb krb5kdc[11746](info): commencing operation
krb5kdc: starting...
W20260928 14:42:14.254364 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:14 dist-test-slave-wwwb krb5kdc[11746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:42:14.374812 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:39949
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:39023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:39949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:14.459226 11759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:14.459476 11759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:14.459503 11759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:14.460731 11759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:14.460780 11759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:14.460796 11759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:14.460840 11759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:14.460865 11759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:14.462481 11759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:39023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:39949
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:39949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:14.462752 11759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:14.462991 11759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:14.468025 11767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:14.468042 11765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:14.468154 11759 server_base.cc:1034] running on GCE node
W20260928 14:42:14.468060 11764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:14.468591 11759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:14.468925 11759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:14.470095 11759 hybrid_clock.cc:648] HybridClock initialized: now 1790606534470096 us; error 46 us; skew 500 ppm
Sep 28 14:42:14 dist-test-slave-wwwb krb5kdc[11746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606534, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:14.472750 11759 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:42:14.473168 11759 webserver.cc:460] Webserver started at http://127.22.13.126:39945/ using document root <none> and password file <none>
I20260928 14:42:14.473413 11759 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:14.473461 11759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:14.473639 11759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:14.474839 11759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "07eeaf99b64c4bc09f299e8249f42d9d"
format_stamp: "Formatted at 2026-09-28 14:42:14 on dist-test-slave-wwwb"
server_key: "e7bf10f42cbfce16b35c4a59391a6b72"
server_key_iv: "0d4394ad00359bab14c56b107d23e8b4"
server_key_version: "encryptionkey@0"
I20260928 14:42:14.475221 11759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "07eeaf99b64c4bc09f299e8249f42d9d"
format_stamp: "Formatted at 2026-09-28 14:42:14 on dist-test-slave-wwwb"
server_key: "e7bf10f42cbfce16b35c4a59391a6b72"
server_key_iv: "0d4394ad00359bab14c56b107d23e8b4"
server_key_version: "encryptionkey@0"
I20260928 14:42:14.476609 11759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:42:14.477380 11774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:14.477577 11759 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:14.477676 11759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "07eeaf99b64c4bc09f299e8249f42d9d"
format_stamp: "Formatted at 2026-09-28 14:42:14 on dist-test-slave-wwwb"
server_key: "e7bf10f42cbfce16b35c4a59391a6b72"
server_key_iv: "0d4394ad00359bab14c56b107d23e8b4"
server_key_version: "encryptionkey@0"
I20260928 14:42:14.477761 11759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:14.491286 11759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:14.493883 11759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:14.494148 11759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:14.498183 11759 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:39949
I20260928 14:42:14.498255 11826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:39949 every 8 connection(s)
I20260928 14:42:14.498625 11759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:14.499605 11827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:14.499928 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 11759
I20260928 14:42:14.500054 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:14.500252 22581 external_mini_cluster.cc:1428] Setting key cd953ade0695e43c9976607313304158
I20260928 14:42:14.502645 11827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d: Bootstrap starting.
I20260928 14:42:14.503420 11827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:14.503738 11827 log.cc:824] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:14.504549 11827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d: No bootstrap required, opened a new log
I20260928 14:42:14.506560 11830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:14.500831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43033 (local address 127.22.13.126:39949)
0928 14:42:14.501020 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:14.501029 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:14.501056 (+ 27us) server_negotiation.cc:407] Connection header received
0928 14:42:14.501500 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:14.501507 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:14.501559 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:14.501693 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:14.502005 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:14.502977 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:14.504136 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:14.504637 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:14.504788 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:14.505629 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:14.505633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:14.505634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:14.505878 (+ 244us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:14.505937 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:14.506016 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:14.506133 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:14.506261 (+ 128us) server_negotiation.cc:299] Negotiation successful
0928 14:42:14.506292 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260928 14:42:14.507035 11827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07eeaf99b64c4bc09f299e8249f42d9d" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39949 } }
I20260928 14:42:14.507225 11827 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:14.507274 11827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07eeaf99b64c4bc09f299e8249f42d9d, State: Initialized, Role: FOLLOWER
I20260928 14:42:14.507427 11827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07eeaf99b64c4bc09f299e8249f42d9d" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39949 } }
I20260928 14:42:14.507527 11827 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:14.507586 11827 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:14.507670 11827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:14.507690 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:14.508287 11827 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07eeaf99b64c4bc09f299e8249f42d9d" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39949 } }
I20260928 14:42:14.508375 11827 leader_election.cc:302] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07eeaf99b64c4bc09f299e8249f42d9d; no voters:
I20260928 14:42:14.508610 11827 leader_election.cc:288] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:14.508644 11832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:14.508862 11832 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [term 1 LEADER]: Becoming Leader. State: Replica: 07eeaf99b64c4bc09f299e8249f42d9d, State: Running, Role: LEADER
I20260928 14:42:14.508960 11827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:14.508994 11832 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07eeaf99b64c4bc09f299e8249f42d9d" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39949 } }
I20260928 14:42:14.509348 11832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07eeaf99b64c4bc09f299e8249f42d9d. Latest consensus state: current_term: 1 leader_uuid: "07eeaf99b64c4bc09f299e8249f42d9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07eeaf99b64c4bc09f299e8249f42d9d" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39949 } } }
I20260928 14:42:14.509377 11836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07eeaf99b64c4bc09f299e8249f42d9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07eeaf99b64c4bc09f299e8249f42d9d" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39949 } } }
I20260928 14:42:14.509516 11836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:14.509716 11832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:14.509861 11845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:14.510419 11845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:14.512401 11845 catalog_manager.cc:1347] Generated new cluster ID: 0ea9d88b01824ab5b79136de2af7a0ed
I20260928 14:42:14.512457 11845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:42:14.518669 11830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:14.509305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53113 (local address 127.22.13.126:39949)
0928 14:42:14.509341 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:14.509343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:14.509352 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:42:14.509448 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:14.509449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:14.509459 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:14.509491 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:14.509716 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:14.510526 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:14.511525 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:14.511817 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:14.511909 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:14.518348 (+ 6439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:14.518350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:14.518350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:14.518357 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:14.518376 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:14.518381 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:14.518466 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:14.518535 (+ 69us) server_negotiation.cc:299] Negotiation successful
0928 14:42:14.518548 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 14:42:14.520493 11845 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:14.521024 11845 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:14.538728 11845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07eeaf99b64c4bc09f299e8249f42d9d: Generated new TSK 0
I20260928 14:42:14.538981 11845 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:14.542762 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 11759
W20260928 14:42:14.547170 11744 connection.cc:535] client connection to 127.22.13.126:39949 recv error: Network error: failed to read from TLS socket (remote: 127.22.13.126:39949): Connection reset by peer (error 104)
2026-09-28T14:42:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1315 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:14 dist-test-slave-wwwb krb5kdc[11858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:14 dist-test-slave-wwwb krb5kdc[11858](info): set up 2 sockets
Sep 28 14:42:14 dist-test-slave-wwwb krb5kdc[11858](info): commencing operation
krb5kdc: starting...
W20260928 14:42:15.574496 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:15 dist-test-slave-wwwb krb5kdc[11858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:42:15.696406 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41783
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:39321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:41783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:15.780893 11871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:15.781172 11871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:15.781216 11871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:15.782612 11871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:15.782696 11871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:15.782721 11871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:15.782753 11871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:15.782781 11871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:15.784399 11871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:39321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:41783
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:15.784715 11871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:15.784957 11871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:15.790041 11876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:15.790035 11879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:15.790035 11877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:15.790328 11871 server_base.cc:1034] running on GCE node
I20260928 14:42:15.790570 11871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:15.790827 11871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:15.791972 11871 hybrid_clock.cc:648] HybridClock initialized: now 1790606535791960 us; error 28 us; skew 500 ppm
Sep 28 14:42:15 dist-test-slave-wwwb krb5kdc[11858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606535, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:15.794605 11871 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:42:15.795133 11871 webserver.cc:460] Webserver started at http://127.22.13.126:44951/ using document root <none> and password file <none>
I20260928 14:42:15.795346 11871 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:15.795387 11871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:15.795477 11871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:15.796535 11871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "13104befe29f4f24bcdf297bef3631b3"
format_stamp: "Formatted at 2026-09-28 14:42:15 on dist-test-slave-wwwb"
server_key: "d3d44de6d5b6b3379fff22d3349fb16b"
server_key_iv: "523fa12c5428efc0932652595030d24d"
server_key_version: "encryptionkey@0"
I20260928 14:42:15.796845 11871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "13104befe29f4f24bcdf297bef3631b3"
format_stamp: "Formatted at 2026-09-28 14:42:15 on dist-test-slave-wwwb"
server_key: "d3d44de6d5b6b3379fff22d3349fb16b"
server_key_iv: "523fa12c5428efc0932652595030d24d"
server_key_version: "encryptionkey@0"
I20260928 14:42:15.798220 11871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 14:42:15.798938 11886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:15.799129 11871 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:42:15.799217 11871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "13104befe29f4f24bcdf297bef3631b3"
format_stamp: "Formatted at 2026-09-28 14:42:15 on dist-test-slave-wwwb"
server_key: "d3d44de6d5b6b3379fff22d3349fb16b"
server_key_iv: "523fa12c5428efc0932652595030d24d"
server_key_version: "encryptionkey@0"
I20260928 14:42:15.799278 11871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:15.802626 11871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:15.804574 11871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:15.804687 11871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:15.808519 11871 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:41783
I20260928 14:42:15.808573 11938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:41783 every 8 connection(s)
I20260928 14:42:15.808936 11871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:15.810081 11939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:15.811851 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 11871
I20260928 14:42:15.811978 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:15.812180 22581 external_mini_cluster.cc:1428] Setting key f9fe67ccff9c991db5d508f91eb59b41
I20260928 14:42:15.812990 11939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3: Bootstrap starting.
I20260928 14:42:15.813665 11939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:15.814107 11939 log.cc:824] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:15.814987 11942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:15.812752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49059 (local address 127.22.13.126:41783)
0928 14:42:15.812937 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:15.812945 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:15.812972 (+ 27us) server_negotiation.cc:407] Connection header received
0928 14:42:15.813457 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:15.813465 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:15.813493 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:15.813624 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:15.814477 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:15.814481 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:15.814481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:15.814486 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:15.814533 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:15.814547 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:15.814627 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:15.814727 (+ 100us) server_negotiation.cc:299] Negotiation successful
0928 14:42:15.814789 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260928 14:42:15.815006 11939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3: No bootstrap required, opened a new log
I20260928 14:42:15.816005 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:15.817698 11939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13104befe29f4f24bcdf297bef3631b3" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41783 } }
I20260928 14:42:15.817927 11939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:15.818013 11939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13104befe29f4f24bcdf297bef3631b3, State: Initialized, Role: FOLLOWER
I20260928 14:42:15.818156 11939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13104befe29f4f24bcdf297bef3631b3" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41783 } }
I20260928 14:42:15.818275 11939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:15.818336 11939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:15.818393 11939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:15.818807 11942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:15.817666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51687 (local address 127.22.13.126:41783)
0928 14:42:15.817708 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:15.817710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:15.817824 (+ 114us) server_negotiation.cc:407] Connection header received
0928 14:42:15.817851 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:15.817851 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:15.817857 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:15.817927 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:15.818599 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:15.818601 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:15.818601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:15.818605 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:15.818622 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:15.818628 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:15.818696 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:15.818737 (+ 41us) server_negotiation.cc:299] Negotiation successful
0928 14:42:15.818751 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:42:15.819348 11939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13104befe29f4f24bcdf297bef3631b3" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41783 } }
I20260928 14:42:15.819463 11939 leader_election.cc:302] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13104befe29f4f24bcdf297bef3631b3; no voters:
I20260928 14:42:15.819670 11939 leader_election.cc:288] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:15.819712 11949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:15.819949 11949 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [term 1 LEADER]: Becoming Leader. State: Replica: 13104befe29f4f24bcdf297bef3631b3, State: Running, Role: LEADER
I20260928 14:42:15.819963 11939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:15.820104 11949 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13104befe29f4f24bcdf297bef3631b3" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41783 } }
I20260928 14:42:15.820505 11950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13104befe29f4f24bcdf297bef3631b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13104befe29f4f24bcdf297bef3631b3" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41783 } } }
I20260928 14:42:15.820515 11951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13104befe29f4f24bcdf297bef3631b3. Latest consensus state: current_term: 1 leader_uuid: "13104befe29f4f24bcdf297bef3631b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13104befe29f4f24bcdf297bef3631b3" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41783 } } }
I20260928 14:42:15.820665 11950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:15.820673 11951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:15.820966 11958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:15.821489 11958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:15.823151 11958 catalog_manager.cc:1347] Generated new cluster ID: 04b9a3aea2a245d7a1594c44bf91141d
I20260928 14:42:15.823222 11958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:42:15.829267 11958 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:15.829460 11958 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:15.833894 11958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13104befe29f4f24bcdf297bef3631b3: Generated new TSK 0
I20260928 14:42:15.834136 11958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:15.839371 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 11871
2026-09-28T14:42:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:15 dist-test-slave-wwwb krb5kdc[11970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:15 dist-test-slave-wwwb krb5kdc[11970](info): set up 2 sockets
Sep 28 14:42:15 dist-test-slave-wwwb krb5kdc[11970](info): commencing operation
krb5kdc: starting...
W20260928 14:42:16.870291 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:16 dist-test-slave-wwwb krb5kdc[11970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:42:16.990621 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:42049
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:34935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:42049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:17.078296 11983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:17.078648 11983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:17.078696 11983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:17.080946 11983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:17.081034 11983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:17.081061 11983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:17.081084 11983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:17.081103 11983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:17.084101 11983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:34935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:42049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:42049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:17.084432 11983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:17.084728 11983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:17.090314 11989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:17.090314 11991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:17.090508 11988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:17.090794 11983 server_base.cc:1034] running on GCE node
I20260928 14:42:17.091075 11983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:17.091444 11983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:17.092649 11983 hybrid_clock.cc:648] HybridClock initialized: now 1790606537092619 us; error 44 us; skew 500 ppm
Sep 28 14:42:17 dist-test-slave-wwwb krb5kdc[11970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606537, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:17.096267 11983 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:42:17.096962 11983 webserver.cc:460] Webserver started at http://127.22.13.126:42327/ using document root <none> and password file <none>
I20260928 14:42:17.097213 11983 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:17.097291 11983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:17.097445 11983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:17.099081 11983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "1ce9a4093470418ca0089c5a8d7b973a"
format_stamp: "Formatted at 2026-09-28 14:42:17 on dist-test-slave-wwwb"
server_key: "31b9287aec7b1a7a5b1b80b03e93927a"
server_key_iv: "d9b371f6f63f1ebc486b3db329db70ab"
server_key_version: "encryptionkey@0"
I20260928 14:42:17.099581 11983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "1ce9a4093470418ca0089c5a8d7b973a"
format_stamp: "Formatted at 2026-09-28 14:42:17 on dist-test-slave-wwwb"
server_key: "31b9287aec7b1a7a5b1b80b03e93927a"
server_key_iv: "d9b371f6f63f1ebc486b3db329db70ab"
server_key_version: "encryptionkey@0"
I20260928 14:42:17.101670 11983 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260928 14:42:17.102775 11998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:17.102993 11983 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:17.103112 11983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "1ce9a4093470418ca0089c5a8d7b973a"
format_stamp: "Formatted at 2026-09-28 14:42:17 on dist-test-slave-wwwb"
server_key: "31b9287aec7b1a7a5b1b80b03e93927a"
server_key_iv: "d9b371f6f63f1ebc486b3db329db70ab"
server_key_version: "encryptionkey@0"
I20260928 14:42:17.103214 11983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:17.131536 11983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:17.134244 11983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:17.134428 11983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:17.138911 11983 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:42049
I20260928 14:42:17.139345 11983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:17.139321 12050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:42049 every 8 connection(s)
I20260928 14:42:17.140060 12051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:17.142619 12051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a: Bootstrap starting.
I20260928 14:42:17.143263 12051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:17.143563 12051 log.cc:824] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:17.144381 12051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a: No bootstrap required, opened a new log
I20260928 14:42:17.146188 12051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce9a4093470418ca0089c5a8d7b973a" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42049 } }
I20260928 14:42:17.146375 12051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:17.146425 12051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ce9a4093470418ca0089c5a8d7b973a, State: Initialized, Role: FOLLOWER
I20260928 14:42:17.146484 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 11983
I20260928 14:42:17.146646 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:17.146636 12051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce9a4093470418ca0089c5a8d7b973a" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42049 } }
I20260928 14:42:17.146737 12051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:17.146785 12051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:17.146842 12051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:17.146849 22581 external_mini_cluster.cc:1428] Setting key 1b930250c65130507131aa9a14b9b850
I20260928 14:42:17.148248 12051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce9a4093470418ca0089c5a8d7b973a" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42049 } }
I20260928 14:42:17.148375 12051 leader_election.cc:302] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ce9a4093470418ca0089c5a8d7b973a; no voters:
I20260928 14:42:17.148562 12051 leader_election.cc:288] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:17.148598 12056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:17.148705 12056 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [term 1 LEADER]: Becoming Leader. State: Replica: 1ce9a4093470418ca0089c5a8d7b973a, State: Running, Role: LEADER
I20260928 14:42:17.148789 12056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce9a4093470418ca0089c5a8d7b973a" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42049 } }
I20260928 14:42:17.148857 12051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:17.149138 12058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ce9a4093470418ca0089c5a8d7b973a. Latest consensus state: current_term: 1 leader_uuid: "1ce9a4093470418ca0089c5a8d7b973a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce9a4093470418ca0089c5a8d7b973a" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42049 } } }
I20260928 14:42:17.149266 12058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:17.149137 12057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ce9a4093470418ca0089c5a8d7b973a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ce9a4093470418ca0089c5a8d7b973a" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42049 } } }
I20260928 14:42:17.149631 12057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:17.149997 12062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:17.150586 12062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:17.152267 12062 catalog_manager.cc:1347] Generated new cluster ID: c4f9f37d20b94e47a0f42faf7a5ec60a
I20260928 14:42:17.152326 12062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:42:17 dist-test-slave-wwwb krb5kdc[11970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:42:17.160272 12055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:17.147438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44451 (local address 127.22.13.126:42049)
0928 14:42:17.147698 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:17.147714 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:17.147766 (+ 52us) server_negotiation.cc:407] Connection header received
0928 14:42:17.147923 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:17.147931 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:17.147995 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:17.148162 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:17.149027 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:17.150642 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:17.152034 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:17.152644 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:17.152757 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:17.155960 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:17.155965 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:17.155966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:17.155982 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:17.158288 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:17.158754 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:17.158759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:17.158759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:17.158957 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:17.159208 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:17.159210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:17.159210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:17.159608 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:17.159792 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:17.159877 (+ 85us) server_negotiation.cc:299] Negotiation successful
0928 14:42:17.160043 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":117,"threads_started":1}
I20260928 14:42:17.161410 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:17.162307 12062 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:17.163013 12062 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:17.174190 12055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:17.166342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50827 (local address 127.22.13.126:42049)
0928 14:42:17.166392 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:17.166394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:17.166490 (+ 96us) server_negotiation.cc:407] Connection header received
0928 14:42:17.166534 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:17.166535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:17.166548 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:17.166633 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:17.167304 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:17.168096 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:17.169196 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:17.169531 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:17.169619 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:17.171119 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:17.171121 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:17.171121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:17.171130 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:17.172888 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:17.173301 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:17.173303 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:17.173303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:17.173376 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:17.173586 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:17.173587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:17.173587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:17.173735 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:17.173847 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:17.173986 (+ 139us) server_negotiation.cc:299] Negotiation successful
0928 14:42:17.174086 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 14:42:17.179188 12062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ce9a4093470418ca0089c5a8d7b973a: Generated new TSK 0
I20260928 14:42:17.179425 12062 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:17.182174 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 11983
2026-09-28T14:42:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1342 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:17 dist-test-slave-wwwb krb5kdc[12082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:17 dist-test-slave-wwwb krb5kdc[12082](info): set up 2 sockets
Sep 28 14:42:17 dist-test-slave-wwwb krb5kdc[12082](info): commencing operation
krb5kdc: starting...
W20260928 14:42:18.214531 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:18 dist-test-slave-wwwb krb5kdc[12082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:42:18.333287 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41733
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:41733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:18.416146 12095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:18.416420 12095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:18.416465 12095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:18.418023 12095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:18.418105 12095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:18.418130 12095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:18.418152 12095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:18.418180 12095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:18.420508 12095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:41733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:41733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:18.420807 12095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:18.421094 12095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:18.426350 12100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:18.426466 12103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:18.426362 12101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:18.426578 12095 server_base.cc:1034] running on GCE node
I20260928 14:42:18.426839 12095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:18.427165 12095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:18.428323 12095 hybrid_clock.cc:648] HybridClock initialized: now 1790606538428308 us; error 29 us; skew 500 ppm
Sep 28 14:42:18 dist-test-slave-wwwb krb5kdc[12082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606538, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:18.438678 12095 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:42:18.439301 12095 webserver.cc:460] Webserver started at http://127.22.13.126:35347/ using document root <none> and password file <none>
I20260928 14:42:18.439567 12095 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:18.439623 12095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:18.439817 12095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:18.441053 12095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "e414f9184f7c475ba9d0b57943a13b4b"
format_stamp: "Formatted at 2026-09-28 14:42:18 on dist-test-slave-wwwb"
server_key: "ed7228872bf9c07d7345f9824a267ff5"
server_key_iv: "46298fa2645cbfa58d794df1ae24f0a5"
server_key_version: "encryptionkey@0"
I20260928 14:42:18.441478 12095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "e414f9184f7c475ba9d0b57943a13b4b"
format_stamp: "Formatted at 2026-09-28 14:42:18 on dist-test-slave-wwwb"
server_key: "ed7228872bf9c07d7345f9824a267ff5"
server_key_iv: "46298fa2645cbfa58d794df1ae24f0a5"
server_key_version: "encryptionkey@0"
I20260928 14:42:18.443110 12095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260928 14:42:18.444010 12110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:18.444232 12095 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:18.444340 12095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "e414f9184f7c475ba9d0b57943a13b4b"
format_stamp: "Formatted at 2026-09-28 14:42:18 on dist-test-slave-wwwb"
server_key: "ed7228872bf9c07d7345f9824a267ff5"
server_key_iv: "46298fa2645cbfa58d794df1ae24f0a5"
server_key_version: "encryptionkey@0"
I20260928 14:42:18.444420 12095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:18.460318 12095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:18.462503 12095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:18.462672 12095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:18.466828 12095 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:41733
I20260928 14:42:18.466863 12162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:41733 every 8 connection(s)
I20260928 14:42:18.467265 12095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:18.468163 12163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:18.468400 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 12095
I20260928 14:42:18.468518 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:18.468691 22581 external_mini_cluster.cc:1428] Setting key c75802ad01d3ea57596fd3a8600c55df
I20260928 14:42:18.471555 12163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b: Bootstrap starting.
I20260928 14:42:18.472330 12163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:18.472657 12163 log.cc:824] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:18.473570 12163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b: No bootstrap required, opened a new log
I20260928 14:42:18.474759 12166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:18.469246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43871 (local address 127.22.13.126:41733)
0928 14:42:18.469445 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:18.469453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:18.469479 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:42:18.469920 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:18.469927 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:18.470028 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:18.470145 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:18.470382 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:18.471245 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:18.472470 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:18.473087 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:18.473177 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:18.474021 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:18.474026 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:18.474026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:18.474256 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:18.474297 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:18.474312 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:18.474384 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:18.474450 (+ 66us) server_negotiation.cc:299] Negotiation successful
0928 14:42:18.474522 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":144,"thread_start_us":111,"threads_started":1}
I20260928 14:42:18.475687 12163 raft_consensus.cc:348] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e414f9184f7c475ba9d0b57943a13b4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41733 } }
I20260928 14:42:18.475891 12163 raft_consensus.cc:374] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:18.475929 12163 raft_consensus.cc:729] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e414f9184f7c475ba9d0b57943a13b4b, State: Initialized, Role: FOLLOWER
I20260928 14:42:18.476042 12163 consensus_queue.cc:260] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e414f9184f7c475ba9d0b57943a13b4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41733 } }
I20260928 14:42:18.476141 12163 raft_consensus.cc:388] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:18.476178 12163 raft_consensus.cc:482] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:18.476228 12163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:18.476281 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:18.476827 12163 raft_consensus.cc:504] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e414f9184f7c475ba9d0b57943a13b4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41733 } }
I20260928 14:42:18.476908 12163 leader_election.cc:302] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e414f9184f7c475ba9d0b57943a13b4b; no voters:
I20260928 14:42:18.477068 12163 leader_election.cc:288] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:18.477101 12168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:18.477206 12168 raft_consensus.cc:686] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [term 1 LEADER]: Becoming Leader. State: Replica: e414f9184f7c475ba9d0b57943a13b4b, State: Running, Role: LEADER
I20260928 14:42:18.477294 12168 consensus_queue.cc:237] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e414f9184f7c475ba9d0b57943a13b4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41733 } }
I20260928 14:42:18.477324 12163 sys_catalog.cc:564] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:18.477586 12168 sys_catalog.cc:455] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e414f9184f7c475ba9d0b57943a13b4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e414f9184f7c475ba9d0b57943a13b4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41733 } } }
I20260928 14:42:18.477648 12168 sys_catalog.cc:458] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:18.477762 12172 sys_catalog.cc:455] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e414f9184f7c475ba9d0b57943a13b4b. Latest consensus state: current_term: 1 leader_uuid: "e414f9184f7c475ba9d0b57943a13b4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e414f9184f7c475ba9d0b57943a13b4b" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 41733 } } }
I20260928 14:42:18.477869 12172 sys_catalog.cc:458] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:18.478329 12176 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:18.478835 12176 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:18.481586 12166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:18.477660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50971 (local address 127.22.13.126:41733)
0928 14:42:18.477699 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:18.477701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:18.478159 (+ 458us) server_negotiation.cc:407] Connection header received
0928 14:42:18.478230 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:18.478230 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:18.478241 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:18.478303 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:18.478593 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:18.479378 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:18.480276 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:18.480493 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:18.480583 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:18.481321 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:18.481323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:18.481323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:18.481328 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:18.481343 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:18.481347 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:18.481416 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:18.481475 (+ 59us) server_negotiation.cc:299] Negotiation successful
0928 14:42:18.481500 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:42:18.483099 12176 catalog_manager.cc:1347] Generated new cluster ID: 08ca1ade132642618a52f10139392f0d
I20260928 14:42:18.483165 12176 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:42:18.504685 12176 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:18.505286 12176 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:18.509370 12176 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e414f9184f7c475ba9d0b57943a13b4b: Generated new TSK 0
I20260928 14:42:18.509591 12176 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:18.514780 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 12095
2026-09-28T14:42:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1331 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:18 dist-test-slave-wwwb krb5kdc[12194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:18 dist-test-slave-wwwb krb5kdc[12194](info): set up 2 sockets
Sep 28 14:42:18 dist-test-slave-wwwb krb5kdc[12194](info): commencing operation
krb5kdc: starting...
W20260928 14:42:19.550518 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:19 dist-test-slave-wwwb krb5kdc[12194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:42:19.672286 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:38465
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:46107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:38465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:19.755722 12207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:19.755973 12207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:19.755999 12207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:19.757282 12207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:19.757331 12207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:19.757347 12207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:19.757360 12207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:19.757373 12207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:19.758965 12207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:46107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:38465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:38465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:19.759189 12207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:19.759367 12207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:19.764262 12212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:19.764258 12215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:19.764262 12213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:19.764613 12207 server_base.cc:1034] running on GCE node
I20260928 14:42:19.764894 12207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:19.765240 12207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:19.766408 12207 hybrid_clock.cc:648] HybridClock initialized: now 1790606539766402 us; error 38 us; skew 500 ppm
Sep 28 14:42:19 dist-test-slave-wwwb krb5kdc[12194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606539, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:19.768998 12207 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:42:19.769449 12207 webserver.cc:460] Webserver started at http://127.22.13.126:37907/ using document root <none> and password file <none>
I20260928 14:42:19.769693 12207 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:19.769776 12207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:19.769917 12207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:19.771051 12207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "92449371995e4c53ba04b8e97b04ad65"
format_stamp: "Formatted at 2026-09-28 14:42:19 on dist-test-slave-wwwb"
server_key: "5ae9d8616ed37a511a96917129136719"
server_key_iv: "4e799bc5a7b36b72c9fbde2a9e5b840f"
server_key_version: "encryptionkey@0"
I20260928 14:42:19.771449 12207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "92449371995e4c53ba04b8e97b04ad65"
format_stamp: "Formatted at 2026-09-28 14:42:19 on dist-test-slave-wwwb"
server_key: "5ae9d8616ed37a511a96917129136719"
server_key_iv: "4e799bc5a7b36b72c9fbde2a9e5b840f"
server_key_version: "encryptionkey@0"
I20260928 14:42:19.772814 12207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:42:19.773573 12222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:19.773753 12207 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:42:19.773862 12207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "92449371995e4c53ba04b8e97b04ad65"
format_stamp: "Formatted at 2026-09-28 14:42:19 on dist-test-slave-wwwb"
server_key: "5ae9d8616ed37a511a96917129136719"
server_key_iv: "4e799bc5a7b36b72c9fbde2a9e5b840f"
server_key_version: "encryptionkey@0"
I20260928 14:42:19.773942 12207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:19.777436 12207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:19.779927 12207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:19.780104 12207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:19.784463 12207 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:38465
I20260928 14:42:19.784498 12274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:38465 every 8 connection(s)
I20260928 14:42:19.784875 12207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:19.785912 12275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:19.787431 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 12207
I20260928 14:42:19.787577 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:19.787752 22581 external_mini_cluster.cc:1428] Setting key 70c3f24b44f9507b30bcbb5b03394d33
I20260928 14:42:19.788794 12275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65: Bootstrap starting.
I20260928 14:42:19.789553 12275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:19.789867 12275 log.cc:824] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:19.790647 12278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:19.788315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53237 (local address 127.22.13.126:38465)
0928 14:42:19.788518 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:19.788529 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:19.788564 (+ 35us) server_negotiation.cc:407] Connection header received
0928 14:42:19.789152 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:19.789162 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:19.789204 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:19.789368 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:19.790132 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:19.790135 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:19.790136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:19.790140 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:19.790190 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:19.790201 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:19.790283 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:19.790371 (+ 88us) server_negotiation.cc:299] Negotiation successful
0928 14:42:19.790441 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":86,"threads_started":1}
I20260928 14:42:19.790751 12275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65: No bootstrap required, opened a new log
I20260928 14:42:19.791738 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:19.793583 12275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92449371995e4c53ba04b8e97b04ad65" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 38465 } }
I20260928 14:42:19.793807 12275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:19.793866 12275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92449371995e4c53ba04b8e97b04ad65, State: Initialized, Role: FOLLOWER
I20260928 14:42:19.794036 12275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92449371995e4c53ba04b8e97b04ad65" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 38465 } }
I20260928 14:42:19.794157 12275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:19.794220 12275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:19.794284 12275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:19.794719 12278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:19.793504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52151 (local address 127.22.13.126:38465)
0928 14:42:19.793556 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:19.793558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:19.793680 (+ 122us) server_negotiation.cc:407] Connection header received
0928 14:42:19.793726 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:19.793726 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:19.793731 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:19.793780 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:19.794471 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:19.794472 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:19.794472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:19.794477 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:19.794493 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:19.794497 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:19.794555 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:19.794619 (+ 64us) server_negotiation.cc:299] Negotiation successful
0928 14:42:19.794635 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 14:42:19.795009 12275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92449371995e4c53ba04b8e97b04ad65" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 38465 } }
I20260928 14:42:19.795115 12275 leader_election.cc:302] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92449371995e4c53ba04b8e97b04ad65; no voters:
I20260928 14:42:19.795310 12275 leader_election.cc:288] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:19.795360 12285 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:19.795535 12285 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [term 1 LEADER]: Becoming Leader. State: Replica: 92449371995e4c53ba04b8e97b04ad65, State: Running, Role: LEADER
I20260928 14:42:19.795575 12275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:19.795701 12285 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92449371995e4c53ba04b8e97b04ad65" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 38465 } }
I20260928 14:42:19.796070 12287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92449371995e4c53ba04b8e97b04ad65. Latest consensus state: current_term: 1 leader_uuid: "92449371995e4c53ba04b8e97b04ad65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92449371995e4c53ba04b8e97b04ad65" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 38465 } } }
I20260928 14:42:19.796069 12286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92449371995e4c53ba04b8e97b04ad65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92449371995e4c53ba04b8e97b04ad65" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 38465 } } }
I20260928 14:42:19.796227 12287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:19.796237 12286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:19.796471 12293 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:19.797004 12293 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:19.798959 12293 catalog_manager.cc:1347] Generated new cluster ID: 873a73f4f6ea4228b41a037eca240064
I20260928 14:42:19.799026 12293 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:42:19.805151 12293 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:19.805367 12293 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:19.813151 12293 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92449371995e4c53ba04b8e97b04ad65: Generated new TSK 0
I20260928 14:42:19.813388 12293 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:19.817533 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 12207
2026-09-28T14:42:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1303 ms)
[----------] 18 tests from AuthTokenIssuingTest (23729 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:19 dist-test-slave-wwwb krb5kdc[12306](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:19 dist-test-slave-wwwb krb5kdc[12306](info): set up 2 sockets
Sep 28 14:42:19 dist-test-slave-wwwb krb5kdc[12306](info): commencing operation
krb5kdc: starting...
W20260928 14:42:20.846702 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:20 dist-test-slave-wwwb krb5kdc[12306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:42:20.969132 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:42527
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:42527,127.22.13.125:33413,127.22.13.124:44037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:21.052083 12319 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:42:21.052392 12319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:21.052425 12319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:21.052470 12319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:21.053673 12319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:21.053752 12319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:21.053776 12319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:21.053797 12319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:21.053828 12319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:21.055401 12319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:42527,127.22.13.125:33413,127.22.13.124:44037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:42527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:21.055706 12319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:21.055989 12319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:21.060998 12325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:21.060989 12324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:21.060989 12327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:21.061200 12319 server_base.cc:1034] running on GCE node
I20260928 14:42:21.061506 12319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:21.061828 12319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:21.063006 12319 hybrid_clock.cc:648] HybridClock initialized: now 1790606541062999 us; error 45 us; skew 500 ppm
Sep 28 14:42:21 dist-test-slave-wwwb krb5kdc[12306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606541, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:21.065606 12319 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:42:21.066179 12319 webserver.cc:460] Webserver started at http://127.22.13.126:33571/ using document root <none> and password file <none>
I20260928 14:42:21.066437 12319 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:21.066506 12319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:21.066658 12319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:21.067772 12319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "8096efa5cb664498acea9b2e844a24e2"
format_stamp: "Formatted at 2026-09-28 14:42:21 on dist-test-slave-wwwb"
server_key: "e712696f04274158ae65f2c4bbb08c44"
server_key_iv: "042ed7d748c197f639fa699f6c1ab802"
server_key_version: "encryptionkey@0"
I20260928 14:42:21.068142 12319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "8096efa5cb664498acea9b2e844a24e2"
format_stamp: "Formatted at 2026-09-28 14:42:21 on dist-test-slave-wwwb"
server_key: "e712696f04274158ae65f2c4bbb08c44"
server_key_iv: "042ed7d748c197f639fa699f6c1ab802"
server_key_version: "encryptionkey@0"
I20260928 14:42:21.069666 12319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:42:21.070523 12334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:21.070698 12319 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:42:21.070798 12319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "8096efa5cb664498acea9b2e844a24e2"
format_stamp: "Formatted at 2026-09-28 14:42:21 on dist-test-slave-wwwb"
server_key: "e712696f04274158ae65f2c4bbb08c44"
server_key_iv: "042ed7d748c197f639fa699f6c1ab802"
server_key_version: "encryptionkey@0"
I20260928 14:42:21.070885 12319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:21.082080 12319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:21.083963 12319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:21.084136 12319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:21.088579 12319 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:42527
I20260928 14:42:21.088613 12386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:42527 every 8 connection(s)
I20260928 14:42:21.088999 12319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:21.090090 12387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:21.091732 12387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:42:21.094415 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 12319
I20260928 14:42:21.094501 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:21.094671 22581 external_mini_cluster.cc:1428] Setting key cd3843452e0d6b72844fd8ee919aa66e
Sep 28 14:42:21 dist-test-slave-wwwb krb5kdc[12306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606541, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.13.125@KRBTEST.COM; defaulting to no policy
I20260928 14:42:21.103407 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.092454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42527 (local address 127.0.0.1:44500)
0928 14:42:21.092634 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.092677 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:21.093221 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:21.093500 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:21.093502 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:21.093771 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:21.094002 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.094008 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.095129 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.095130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.096311 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.096314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.097050 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.097051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.097108 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.097674 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:21.097684 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:21.099448 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:21.101655 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.101663 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.101665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.101885 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.102289 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.102289 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.102290 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.102378 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.102868 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:21.102874 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:21.102938 (+ 64us) client_negotiation.cc:769] Sending connection context
0928 14:42:21.103011 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 14:42:21.103224 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260928 14:42:21.103470 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.092512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44500 (local address 127.22.13.126:42527)
0928 14:42:21.092673 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.092682 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.092764 (+ 82us) server_negotiation.cc:407] Connection header received
0928 14:42:21.093362 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.093364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.093381 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.093472 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.094103 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.095033 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.096397 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.096931 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.097064 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.099551 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:21.099561 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:21.099561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:21.099570 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:21.101526 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.102079 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.102083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.102083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.102203 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.102462 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.102463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.102463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.102792 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:21.102880 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:21.103105 (+ 225us) server_negotiation.cc:299] Negotiation successful
0928 14:42:21.103339 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
I20260928 14:42:21.104255 12387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:42:21.104620 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.104510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:33413 (local address 127.0.0.1:45486)
0928 14:42:21.104555 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.104600 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260928 14:42:21.104773 12338 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.13.125:33413: connect: Connection refused (error 111)
I20260928 14:42:21.105185 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.105102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:33413 (local address 127.0.0.1:45488)
0928 14:42:21.105144 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.105168 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260928 14:42:21.105331 12387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.125:33413: Network error: Client connection negotiation failed: client connection to 127.22.13.125:33413: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.125.keytab.
Entry for principal kudu/127.22.13.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.125.keytab.
I20260928 14:42:21.120498 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.125:33413
--webserver_interface=127.22.13.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.125.keytab
--principal=kudu/127.22.13.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:42527,127.22.13.125:33413,127.22.13.124:44037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:42:21.154559 12387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } attempt: 1
I20260928 14:42:21.155093 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.154953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:33413 (local address 127.0.0.1:45496)
0928 14:42:21.155019 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.155064 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260928 14:42:21.155426 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.155330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:33413 (local address 127.0.0.1:45498)
0928 14:42:21.155375 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.155402 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 14:42:21.155529 12387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.125:33413: Network error: Client connection negotiation failed: client connection to 127.22.13.125:33413: connect: Connection refused (error 111)
W20260928 14:42:21.201551 12394 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:42:21.201833 12394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:21.201865 12394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:21.201912 12394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:21.203116 12394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:21.203210 12394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:21.203235 12394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:21.203256 12394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:21.203286 12394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:21.204855 12394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:42527,127.22.13.125:33413,127.22.13.124:44037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.125.keytab
--principal=kudu/127.22.13.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.125:33413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.13.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:21.205166 12394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:21.205405 12394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:21.210325 12399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:21.210282 12402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:21.210282 12400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:21.210721 12394 server_base.cc:1034] running on GCE node
I20260928 14:42:21.211009 12394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:21.211305 12394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:21.212450 12394 hybrid_clock.cc:648] HybridClock initialized: now 1790606541212439 us; error 30 us; skew 500 ppm
Sep 28 14:42:21 dist-test-slave-wwwb krb5kdc[12306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606541, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:21.214917 12394 init.cc:377] Logged in from keytab as kudu/127.22.13.125@KRBTEST.COM (short username kudu)
I20260928 14:42:21.215422 12394 webserver.cc:460] Webserver started at http://127.22.13.125:45141/ using document root <none> and password file <none>
I20260928 14:42:21.215619 12394 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:21.215659 12394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:21.215755 12394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:21.216812 12394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/data/instance:
uuid: "5e0c2e22ec78420991fcc70c9e872fa1"
format_stamp: "Formatted at 2026-09-28 14:42:21 on dist-test-slave-wwwb"
server_key: "ba01178bfbf51a35979c9a2cee5169a1"
server_key_iv: "f6386655d4cc9420bc481cfb401f8699"
server_key_version: "encryptionkey@0"
I20260928 14:42:21.217123 12394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/wal/instance:
uuid: "5e0c2e22ec78420991fcc70c9e872fa1"
format_stamp: "Formatted at 2026-09-28 14:42:21 on dist-test-slave-wwwb"
server_key: "ba01178bfbf51a35979c9a2cee5169a1"
server_key_iv: "f6386655d4cc9420bc481cfb401f8699"
server_key_version: "encryptionkey@0"
I20260928 14:42:21.218503 12394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:42:21.219158 12409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:21.219336 12394 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:42:21.219421 12394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/wal
uuid: "5e0c2e22ec78420991fcc70c9e872fa1"
format_stamp: "Formatted at 2026-09-28 14:42:21 on dist-test-slave-wwwb"
server_key: "ba01178bfbf51a35979c9a2cee5169a1"
server_key_iv: "f6386655d4cc9420bc481cfb401f8699"
server_key_version: "encryptionkey@0"
I20260928 14:42:21.219496 12394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:21.223675 12387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } attempt: 2
I20260928 14:42:21.224174 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.224028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:33413 (local address 127.0.0.1:45506)
0928 14:42:21.224090 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.224132 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260928 14:42:21.224526 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.224425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:33413 (local address 127.0.0.1:45514)
0928 14:42:21.224471 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.224505 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:33413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260928 14:42:21.224618 12387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.125:33413: Network error: Client connection negotiation failed: client connection to 127.22.13.125:33413: connect: Connection refused (error 111)
I20260928 14:42:21.227662 12394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:21.229557 12394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:21.229722 12394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:21.233840 12394 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.125:33413
I20260928 14:42:21.233883 12461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.125:33413 every 8 connection(s)
I20260928 14:42:21.234279 12394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/data/info.pb
I20260928 14:42:21.235263 12462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:21.235456 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 12394
I20260928 14:42:21.235551 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-1/wal/instance
I20260928 14:42:21.235760 22581 external_mini_cluster.cc:1428] Setting key 902b3da1d1df301fbdb6b006c47b438b
I20260928 14:42:21.236958 12462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.13.124@KRBTEST.COM; defaulting to no policy
Sep 28 14:42:21 dist-test-slave-wwwb krb5kdc[12306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606541, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.125@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.124@KRBTEST.COM" created.
I20260928 14:42:21.246941 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.237787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44506 (local address 127.22.13.126:42527)
0928 14:42:21.237855 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.237857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.238094 (+ 237us) server_negotiation.cc:407] Connection header received
0928 14:42:21.238558 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.238559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.238573 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.238651 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.239282 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.240080 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.241081 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.241414 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.241492 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.243946 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:21.243948 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:21.243948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:21.243955 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:21.245287 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.245806 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.245807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.245807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.245858 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.246203 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.246205 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.246205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.246365 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:21.246439 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:21.246804 (+ 365us) server_negotiation.cc:299] Negotiation successful
0928 14:42:21.246873 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:42:21.247231 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.237743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42527 (local address 127.0.0.1:44506)
0928 14:42:21.237958 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.238004 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:21.238466 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:21.238722 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:21.238726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:21.238976 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:21.239186 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.239190 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.240182 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.240183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.240993 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.240994 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.241516 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.241517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.241638 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.242261 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:21.242270 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:21.243857 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:21.245431 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.245440 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.245442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.245724 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.245978 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.245979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.245979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.246105 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.246453 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:21.246459 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:21.246659 (+ 200us) client_negotiation.cc:769] Sending connection context
0928 14:42:21.246767 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 14:42:21.246953 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":113,"threads_started":1}
I20260928 14:42:21.247892 12462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:42:21 dist-test-slave-wwwb krb5kdc[12306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606541, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.125@KRBTEST.COM for kudu/127.22.13.125@KRBTEST.COM
I20260928 14:42:21.256377 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.248195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:33413 (local address 127.0.0.1:45526)
0928 14:42:21.248243 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.248254 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:21.248293 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:21.248623 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:21.248624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:21.248737 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:21.248790 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.248792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.250149 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.250150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.250733 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.250734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.251483 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.251484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.251543 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.252005 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:21.252009 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:21.253391 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:21.255312 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.255313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.255314 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.255513 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.255700 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.255701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.255701 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.255809 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.256092 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:21.256093 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:21.256131 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 14:42:21.256183 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 14:42:21.256306 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260928 14:42:21.256549 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.248251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45526 (local address 127.22.13.125:33413)
0928 14:42:21.248463 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.248468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.248492 (+ 24us) server_negotiation.cc:407] Connection header received
0928 14:42:21.248518 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.248520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.248534 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.248616 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.248863 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.250042 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.250817 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.251395 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.251481 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.253480 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:21.253489 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:21.253490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:21.253498 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:21.255219 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.255587 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.255589 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.255590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.255641 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.255894 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.255895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.255895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.256035 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:21.256096 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:21.256231 (+ 135us) server_negotiation.cc:299] Negotiation successful
0928 14:42:21.256368 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":173,"thread_start_us":110,"threads_started":1}
I20260928 14:42:21.257153 12462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:42:21.257511 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.257402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:44037 (local address 127.0.0.1:41444)
0928 14:42:21.257451 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.257492 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:44037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 14:42:21.257673 12411 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.13.124:44037: connect: Connection refused (error 111)
I20260928 14:42:21.258195 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.258101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:44037 (local address 127.0.0.1:41458)
0928 14:42:21.258149 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.258174 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:44037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 14:42:21.258304 12462 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.124:44037: Network error: Client connection negotiation failed: client connection to 127.22.13.124:44037: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.124.keytab.
Entry for principal kudu/127.22.13.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.124.keytab.
I20260928 14:42:21.262058 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.124:44037
--webserver_interface=127.22.13.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.124.keytab
--principal=kudu/127.22.13.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:42527,127.22.13.125:33413,127.22.13.124:44037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:42:21.307601 12462 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } attempt: 1
I20260928 14:42:21.308125 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.307989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:44037 (local address 127.0.0.1:41462)
0928 14:42:21.308054 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.308094 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:44037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260928 14:42:21.308514 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.308385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:44037 (local address 127.0.0.1:41474)
0928 14:42:21.308441 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.308484 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:44037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 14:42:21.308638 12462 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.124:44037: Network error: Client connection negotiation failed: client connection to 127.22.13.124:44037: connect: Connection refused (error 111)
I20260928 14:42:21.315779 12387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } attempt: 3
Sep 28 14:42:21 dist-test-slave-wwwb krb5kdc[12306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606541, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for kudu/127.22.13.125@KRBTEST.COM
I20260928 14:42:21.323741 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.316260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45530 (local address 127.22.13.125:33413)
0928 14:42:21.316321 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.316323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.316378 (+ 55us) server_negotiation.cc:407] Connection header received
0928 14:42:21.316407 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.316408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.316422 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.316476 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.316791 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.317593 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.318400 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.318757 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.318834 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.320854 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:21.320856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:21.320856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:21.320864 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:21.322517 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.322892 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.322894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.322894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.322965 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.323174 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.323175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.323176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.323317 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:21.323393 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:21.323549 (+ 156us) server_negotiation.cc:299] Negotiation successful
0928 14:42:21.323631 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 14:42:21.323733 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.316170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:33413 (local address 127.0.0.1:45530)
0928 14:42:21.316289 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.316301 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:21.316348 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:21.316491 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:21.316492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:21.316641 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:21.316713 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.316715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.317693 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.317694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.318319 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.318321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.318839 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.318840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.318901 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.319378 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:21.319383 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:21.320770 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:21.322601 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.322602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.322603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.322795 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.323051 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.323051 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.323052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.323110 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.323394 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:21.323395 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:21.323443 (+ 48us) client_negotiation.cc:769] Sending connection context
0928 14:42:21.323525 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 14:42:21.323630 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260928 14:42:21.324085 12387 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:42:21.324450 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.324347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:44037 (local address 127.0.0.1:41488)
0928 14:42:21.324397 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.324425 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:44037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 14:42:21.324790 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.324727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:44037 (local address 127.0.0.1:41502)
0928 14:42:21.324759 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.324778 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:44037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260928 14:42:21.324887 12387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.124:44037: Network error: Client connection negotiation failed: client connection to 127.22.13.124:44037: connect: Connection refused (error 111)
W20260928 14:42:21.346351 12469 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:42:21.346576 12469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:21.346594 12469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:21.346616 12469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:21.347759 12469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:21.347800 12469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:21.347815 12469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:21.347826 12469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:21.347839 12469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:21.349278 12469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:42527,127.22.13.125:33413,127.22.13.124:44037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.124.keytab
--principal=kudu/127.22.13.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.124:44037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.13.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:21.349478 12469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:21.349651 12469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:42:21.354871 12469 server_base.cc:1034] running on GCE node
W20260928 14:42:21.354826 12475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:21.354846 12474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:21.354843 12477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:21.355261 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:21.355537 12469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:21.356022 12387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } attempt: 1
I20260928 14:42:21.356441 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.356329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:44037 (local address 127.0.0.1:41518)
0928 14:42:21.356375 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.356411 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:44037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260928 14:42:21.356681 12469 hybrid_clock.cc:648] HybridClock initialized: now 1790606541356670 us; error 37 us; skew 500 ppm
I20260928 14:42:21.356803 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.356698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:44037 (local address 127.0.0.1:41526)
0928 14:42:21.356746 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.356774 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:44037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 14:42:21.356912 12387 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.124:44037: Network error: Client connection negotiation failed: client connection to 127.22.13.124:44037: connect: Connection refused (error 111)
Sep 28 14:42:21 dist-test-slave-wwwb krb5kdc[12306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606541, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:21.359308 12469 init.cc:377] Logged in from keytab as kudu/127.22.13.124@KRBTEST.COM (short username kudu)
I20260928 14:42:21.359795 12469 webserver.cc:460] Webserver started at http://127.22.13.124:34811/ using document root <none> and password file <none>
I20260928 14:42:21.360008 12469 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:21.360049 12469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:21.360144 12469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:21.361203 12469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/data/instance:
uuid: "ac613f5bcd984d64a5b662872f632695"
format_stamp: "Formatted at 2026-09-28 14:42:21 on dist-test-slave-wwwb"
server_key: "ce60b48cc9e802ecad4c546b26d0d809"
server_key_iv: "105d9574cd0a9ca2e4643057642cf8c2"
server_key_version: "encryptionkey@0"
I20260928 14:42:21.361513 12469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/wal/instance:
uuid: "ac613f5bcd984d64a5b662872f632695"
format_stamp: "Formatted at 2026-09-28 14:42:21 on dist-test-slave-wwwb"
server_key: "ce60b48cc9e802ecad4c546b26d0d809"
server_key_iv: "105d9574cd0a9ca2e4643057642cf8c2"
server_key_version: "encryptionkey@0"
I20260928 14:42:21.362933 12469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:42:21.363649 12484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:21.363847 12469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:21.363936 12469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/wal
uuid: "ac613f5bcd984d64a5b662872f632695"
format_stamp: "Formatted at 2026-09-28 14:42:21 on dist-test-slave-wwwb"
server_key: "ce60b48cc9e802ecad4c546b26d0d809"
server_key_iv: "105d9574cd0a9ca2e4643057642cf8c2"
server_key_version: "encryptionkey@0"
I20260928 14:42:21.363997 12469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:21.376780 12462 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } attempt: 2
I20260928 14:42:21.377295 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.377158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:44037 (local address 127.0.0.1:41536)
0928 14:42:21.377220 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.377266 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:44037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 14:42:21.377653 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.377573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:44037 (local address 127.0.0.1:41548)
0928 14:42:21.377615 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.377639 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:44037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260928 14:42:21.377764 12462 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.124:44037: Network error: Client connection negotiation failed: client connection to 127.22.13.124:44037: connect: Connection refused (error 111)
I20260928 14:42:21.387612 12469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:21.389827 12469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:21.390039 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:21.394106 12469 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.124:44037
I20260928 14:42:21.394163 12536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.124:44037 every 8 connection(s)
I20260928 14:42:21.394546 12469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/data/info.pb
I20260928 14:42:21.395536 12537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:21.397001 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 12469
I20260928 14:42:21.397113 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606270465379-22581-0/minicluster-data/master-2/wal/instance
I20260928 14:42:21.397044 12537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:42:21.397305 22581 external_mini_cluster.cc:1428] Setting key e44a9ea6e3c228c687667e410cfaf223
Sep 28 14:42:21 dist-test-slave-wwwb krb5kdc[12306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606541, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.124@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
Sep 28 14:42:21 dist-test-slave-wwwb krb5kdc[12306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.124@KRBTEST.COM
I20260928 14:42:21.407433 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.397798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44518 (local address 127.22.13.126:42527)
0928 14:42:21.397852 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.397854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.398151 (+ 297us) server_negotiation.cc:407] Connection header received
0928 14:42:21.398710 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.398711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.398725 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.398792 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.399375 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.399863 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.400953 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.401249 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.401324 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.403894 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:21.403895 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:21.403896 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:21.403903 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:21.405857 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.406391 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.406392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.406393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.406445 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.406743 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.406745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.406745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.406874 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:21.406949 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:21.407249 (+ 300us) server_negotiation.cc:299] Negotiation successful
0928 14:42:21.407336 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 14:42:21.407629 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.397788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:42527 (local address 127.0.0.1:44518)
0928 14:42:21.398041 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.398082 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:21.398579 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:21.398839 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:21.398841 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:21.399110 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:21.399280 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.399283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.399940 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.399941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.400872 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.400873 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.401392 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.401393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.401531 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.402142 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:21.402150 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:21.403802 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:21.405980 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.405986 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.405989 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.406308 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.406515 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.406516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.406517 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.406655 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.406954 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:21.406962 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:21.407111 (+ 149us) client_negotiation.cc:769] Sending connection context
0928 14:42:21.407220 (+ 109us) client_negotiation.cc:240] Negotiation successful
0928 14:42:21.407382 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":93,"threads_started":1}
I20260928 14:42:21.407871 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.397794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41558 (local address 127.22.13.124:44037)
0928 14:42:21.398045 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.398055 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.398110 (+ 55us) server_negotiation.cc:407] Connection header received
0928 14:42:21.398546 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.398553 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.398573 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.398702 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.399434 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.400449 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.401365 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.401802 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.401856 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.404438 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:21.404447 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:21.404447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:21.404454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:21.406542 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.407000 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.407003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.407004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.407078 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.407293 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.407294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.407294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.407441 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:21.407586 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:21.407679 (+ 93us) server_negotiation.cc:299] Negotiation successful
0928 14:42:21.407758 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":103,"threads_started":1}
I20260928 14:42:21.408440 12537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:42:21 dist-test-slave-wwwb krb5kdc[12306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606541, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.124@KRBTEST.COM for kudu/127.22.13.125@KRBTEST.COM
Sep 28 14:42:21 dist-test-slave-wwwb krb5kdc[12306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:42:21.415782 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.408820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45532 (local address 127.22.13.125:33413)
0928 14:42:21.408859 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.408861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.408870 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:42:21.408896 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.408897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.408909 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.408975 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.409280 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.409897 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.410663 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.410981 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.411045 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.413014 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:21.413015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:21.413015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:21.413023 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:21.414606 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.414998 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.414999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.414999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.415059 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.415275 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.415276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.415276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.415380 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:21.415430 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:21.415603 (+ 173us) server_negotiation.cc:299] Negotiation successful
0928 14:42:21.415684 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:42:21.415782 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.408747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:33413 (local address 127.0.0.1:45532)
0928 14:42:21.408792 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.408804 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:21.408842 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:21.408979 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:21.408980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:21.409141 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:21.409191 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.409193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.409999 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.410000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.410584 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.410586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.411055 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.411056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.411124 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.411573 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:21.411577 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:21.412916 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:21.414688 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.414689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.414689 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.414914 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.415129 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.415130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.415131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.415198 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.415443 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:21.415444 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:21.415499 (+ 55us) client_negotiation.cc:769] Sending connection context
0928 14:42:21.415568 (+ 69us) client_negotiation.cc:240] Negotiation successful
0928 14:42:21.415674 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260928 14:42:21.416577 12537 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:42:21.416678 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.408953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44532 (local address 127.22.13.126:42527)
0928 14:42:21.408993 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.408995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.409002 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:42:21.409023 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.409024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.409032 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.409087 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.409623 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.410176 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.410932 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.411173 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.411246 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.413836 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:21.413837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:21.413837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:21.413843 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:21.415401 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.415753 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.415755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.415755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.415812 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.416126 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.416128 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.416128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.416252 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:21.416450 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:21.416486 (+ 36us) server_negotiation.cc:299] Negotiation successful
0928 14:42:21.416575 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Sep 28 14:42:21 dist-test-slave-wwwb krb5kdc[12306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606541, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.124@KRBTEST.COM for kudu/127.22.13.124@KRBTEST.COM
Sep 28 14:42:21 dist-test-slave-wwwb krb5kdc[12306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.125@KRBTEST.COM
I20260928 14:42:21.424223 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.417031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:44037 (local address 127.0.0.1:41572)
0928 14:42:21.417153 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.417163 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:21.417235 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:21.417367 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:21.417368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:21.417533 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:21.417574 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.417576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.418458 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.418459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.419128 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.419129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.419592 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.419592 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.419631 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.420052 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:21.420059 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:21.421249 (+ 1190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:21.423159 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.423160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.423161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.423382 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.423621 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.423621 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.423622 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.423691 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.423980 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:21.423981 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:21.424015 (+ 34us) client_negotiation.cc:769] Sending connection context
0928 14:42:21.424067 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 14:42:21.424153 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260928 14:42:21.424356 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.417095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41572 (local address 127.22.13.124:44037)
0928 14:42:21.417137 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.417139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.417223 (+ 84us) server_negotiation.cc:407] Connection header received
0928 14:42:21.417289 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.417290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.417298 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.417364 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.417654 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.418386 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.419201 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.419517 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.419595 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.421335 (+ 1740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:21.421336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:21.421336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:21.421344 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:21.423073 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.423478 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.423478 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.423479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.423525 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.423760 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.423761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.423761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.423912 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:21.423991 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:21.424091 (+ 100us) server_negotiation.cc:299] Negotiation successful
0928 14:42:21.424225 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":20}
I20260928 14:42:21.425577 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.417400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45534 (local address 127.22.13.125:33413)
0928 14:42:21.417446 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.417448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.417487 (+ 39us) server_negotiation.cc:407] Connection header received
0928 14:42:21.417533 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.417534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.417543 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.417605 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.418212 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.418848 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.419708 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.419978 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.420064 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.422724 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:21.422726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:21.422726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:21.422733 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:21.424418 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.424797 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.424799 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.424799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.424862 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.425065 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.425066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.425067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.425202 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:21.425371 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:21.425408 (+ 37us) server_negotiation.cc:299] Negotiation successful
0928 14:42:21.425481 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:42:21.426139 12537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695: Bootstrap starting.
I20260928 14:42:21.427110 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:21.427240 12537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:21.427548 12537 log.cc:824] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:21.428396 12537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695: No bootstrap required, opened a new log
I20260928 14:42:21.428560 12361 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8096efa5cb664498acea9b2e844a24e2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44532
I20260928 14:42:21.429926 12537 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } }
I20260928 14:42:21.430166 12537 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac613f5bcd984d64a5b662872f632695, State: Initialized, Role: FOLLOWER
I20260928 14:42:21.430298 12537 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } }
I20260928 14:42:21.430603 12543 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } } }
I20260928 14:42:21.430675 12537 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:21.430699 12543 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:42:21.431977 12387 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } attempt: 2
W20260928 14:42:21.433167 12554 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:42:21.433336 12554 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:42:21.432873 12469 master_runner.cc:186] Error getting master registration for 127.22.13.126:42527: OK, instance_id { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" instance_seqno: 1790606541087193 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:42:21.435354 12469 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 28 14:42:21 dist-test-slave-wwwb krb5kdc[12306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606541, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for kudu/127.22.13.124@KRBTEST.COM
I20260928 14:42:21.442514 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.432489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41580 (local address 127.22.13.124:44037)
0928 14:42:21.432697 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.432700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.435417 (+ 2717us) server_negotiation.cc:407] Connection header received
0928 14:42:21.435444 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.435445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.435456 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.435519 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.435851 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.436433 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.437166 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.437387 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.437465 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.439649 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:21.439650 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:21.439651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:21.439659 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:21.441183 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.441628 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.441628 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.441629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.441680 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.441911 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.441912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.441913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.442096 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:21.442182 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:21.442332 (+ 150us) server_negotiation.cc:299] Negotiation successful
0928 14:42:21.442413 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260928 14:42:21.442514 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.432644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:44037 (local address 127.0.0.1:41580)
0928 14:42:21.434026 (+ 1382us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.434041 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:21.434139 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:21.435528 (+ 1389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:21.435529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:21.435704 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:21.435749 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.435751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.436540 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.436541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.437059 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.437060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.437473 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.437475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.437543 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.438055 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:21.438059 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:21.439524 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:21.441291 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.441292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.441292 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.441545 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.441771 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.441772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.441772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.441846 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.442174 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:21.442175 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:21.442229 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 14:42:21.442303 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 14:42:21.442413 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1359}
I20260928 14:42:21.444000 12387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2: Bootstrap starting.
I20260928 14:42:21.444664 12387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:21.444962 12387 log.cc:824] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:21.445719 12387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2: No bootstrap required, opened a new log
I20260928 14:42:21.447031 12387 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } }
I20260928 14:42:21.447175 12387 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8096efa5cb664498acea9b2e844a24e2, State: Initialized, Role: FOLLOWER
I20260928 14:42:21.447299 12387 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } }
I20260928 14:42:21.447609 12557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } } }
I20260928 14:42:21.447716 12557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:42:21.447739 12361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260928 14:42:21.447784 12361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:21.448447 12361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } }
I20260928 14:42:21.448706 12361 leader_election.cc:288] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [CANDIDATE]: Term 1 election: Requested vote from peers 5e0c2e22ec78420991fcc70c9e872fa1 (127.22.13.125:33413), ac613f5bcd984d64a5b662872f632695 (127.22.13.124:44037)
I20260928 14:42:21.447672 12387 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:21.450418 12511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8096efa5cb664498acea9b2e844a24e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ac613f5bcd984d64a5b662872f632695"
I20260928 14:42:21.450556 12511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:21.451506 12511 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8096efa5cb664498acea9b2e844a24e2 in term 1.
I20260928 14:42:21.452281 12336 leader_election.cc:302] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8096efa5cb664498acea9b2e844a24e2, ac613f5bcd984d64a5b662872f632695; no voters:
I20260928 14:42:21.452430 12557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:21.452581 12557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [term 1 LEADER]: Becoming Leader. State: Replica: 8096efa5cb664498acea9b2e844a24e2, State: Running, Role: LEADER
I20260928 14:42:21.452665 12557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } }
I20260928 14:42:21.453085 12557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8096efa5cb664498acea9b2e844a24e2. Latest consensus state: current_term: 1 leader_uuid: "8096efa5cb664498acea9b2e844a24e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } } }
I20260928 14:42:21.453173 12557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:21.451512 12319 master_runner.cc:186] Error getting master registration for 127.22.13.125:33413: OK, instance_id { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" instance_seqno: 1790606541232672 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:42:21.453838 12319 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260928 14:42:21.454454 12572 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:42:21.454510 12572 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:42:21.454586 12576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:21.454754 12576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:21.458985 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.450016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44544 (local address 127.22.13.126:42527)
0928 14:42:21.450081 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.450084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.450355 (+ 271us) server_negotiation.cc:407] Connection header received
0928 14:42:21.450464 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.450465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.450475 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.450507 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.451403 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.452116 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.453174 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.453451 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.453492 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.455253 (+ 1761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:21.455255 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:21.455255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:21.455262 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:21.457077 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.457813 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.457814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.457815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.457887 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.458222 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.458223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.458224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.458369 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:21.458457 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:21.458792 (+ 335us) server_negotiation.cc:299] Negotiation successful
0928 14:42:21.458891 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:42:21.459398 12511 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695 [term 1 FOLLOWER]: Refusing update from remote peer 8096efa5cb664498acea9b2e844a24e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:21.459708 12557 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:21.461058 12543 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8096efa5cb664498acea9b2e844a24e2. Latest consensus state: current_term: 1 leader_uuid: "8096efa5cb664498acea9b2e844a24e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } } }
I20260928 14:42:21.461169 12543 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:42:21.467005 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.453797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45548 (local address 127.22.13.125:33413)
0928 14:42:21.453843 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.453845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.455761 (+ 1916us) server_negotiation.cc:407] Connection header received
0928 14:42:21.455878 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.455879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.455891 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.455924 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.456613 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.457360 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.461127 (+ 3767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.461980 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.462030 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.463769 (+ 1739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:21.463770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:21.463770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:21.463779 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:21.465577 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.466099 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.466101 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.466101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.466161 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.466388 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.466389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.466390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.466532 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:21.466620 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:21.466818 (+ 198us) server_negotiation.cc:299] Negotiation successful
0928 14:42:21.466907 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:42:21.468075 12557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8096efa5cb664498acea9b2e844a24e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } } }
I20260928 14:42:21.468209 12557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:21.468214 12543 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8096efa5cb664498acea9b2e844a24e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } } }
I20260928 14:42:21.468292 12543 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:42:21.468358 12579 mvcc.cc:205] Tried to move back new op lower bound from 7334324393812983808 to 7334324393790885888. Current Snapshot: MvccSnapshot[applied={T|T < 7334324393812983808}]
I20260928 14:42:21.468497 12581 mvcc.cc:205] Tried to move back new op lower bound from 7334324393812983808 to 7334324393790885888. Current Snapshot: MvccSnapshot[applied={T|T < 7334324393812983808}]
I20260928 14:42:21.468883 12576 catalog_manager.cc:1347] Generated new cluster ID: 589c8c993a2844e68615260ce51ed438
I20260928 14:42:21.468936 12576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:42:21.468906 12462 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } attempt: 3
I20260928 14:42:21.470918 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.455619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41596 (local address 127.22.13.124:44037)
0928 14:42:21.455672 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.455674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.460073 (+ 4399us) server_negotiation.cc:407] Connection header received
0928 14:42:21.460236 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.460237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.460260 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.460291 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.461517 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.462299 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.463796 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.464115 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.464159 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.466513 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:21.466515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:21.466515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:21.466522 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:21.468646 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.469741 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.469742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.469742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.469818 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.470173 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.470174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.470174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.470314 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:21.470386 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:21.470739 (+ 353us) server_negotiation.cc:299] Negotiation successful
0928 14:42:21.470824 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Sep 28 14:42:21 dist-test-slave-wwwb krb5kdc[12306](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606541, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.125@KRBTEST.COM for kudu/127.22.13.124@KRBTEST.COM
I20260928 14:42:21.477289 12576 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:21.478061 12576 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:21.480266 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.469169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:44037 (local address 127.0.0.1:41600)
0928 14:42:21.469253 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:21.469264 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:21.469303 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:21.471101 (+ 1798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:21.471103 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:21.471297 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:21.471348 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.471350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.472261 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.472262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.473108 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:21.473111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.473624 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.473625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:21.473688 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.474268 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:21.474273 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:21.476005 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:21.478827 (+ 2822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.478828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.478829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.479083 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.479416 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:21.479417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:21.479418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:21.479475 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:21.479904 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:21.479905 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:21.479964 (+ 59us) client_negotiation.cc:769] Sending connection context
0928 14:42:21.480020 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 14:42:21.480138 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260928 14:42:21.481252 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:21.469389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41600 (local address 127.22.13.124:44037)
0928 14:42:21.470987 (+ 1598us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.470989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.470997 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:42:21.471017 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.471017 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.471028 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.471095 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.471448 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.472161 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.473208 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:21.473509 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:21.473572 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:21.476144 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:21.476146 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:21.476146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:21.476170 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:21.478683 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.479213 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.479214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.479215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.479300 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:21.479551 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:21.479552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:21.479552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:21.479772 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:21.480787 (+ 1015us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:21.480917 (+ 130us) server_negotiation.cc:299] Negotiation successful
0928 14:42:21.481073 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1585,"thread_start_us":84,"threads_started":1}
I20260928 14:42:21.483223 12462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1: Bootstrap starting.
I20260928 14:42:21.483980 12462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:21.484292 12462 log.cc:824] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:21.485324 12462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1: No bootstrap required, opened a new log
I20260928 14:42:21.487216 12576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2: Generated new TSK 0
I20260928 14:42:21.487537 12576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:21.489111 12462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } }
I20260928 14:42:21.489673 12462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0c2e22ec78420991fcc70c9e872fa1, State: Initialized, Role: FOLLOWER
I20260928 14:42:21.489992 12462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } }
I20260928 14:42:21.493351 12462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:21.493409 12436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8096efa5cb664498acea9b2e844a24e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5e0c2e22ec78420991fcc70c9e872fa1"
I20260928 14:42:21.493552 12436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:21.494532 12436 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8096efa5cb664498acea9b2e844a24e2 in term 1.
I20260928 14:42:21.494760 12435 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1 [term 1 FOLLOWER]: Refusing update from remote peer 8096efa5cb664498acea9b2e844a24e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:21.493361 12589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } } }
I20260928 14:42:21.495159 12584 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:21.495332 12589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1 [sys.catalog]: This master's current role is: FOLLOWER
W20260928 14:42:21.497366 12600 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:42:21.497470 12600 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:42:21.497761 12589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8096efa5cb664498acea9b2e844a24e2. Latest consensus state: current_term: 1 leader_uuid: "8096efa5cb664498acea9b2e844a24e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } } }
I20260928 14:42:21.497886 12589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:42:21.497972 12589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8096efa5cb664498acea9b2e844a24e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } } }
I20260928 14:42:21.498082 12589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:42:21.510725 12601 mvcc.cc:205] Tried to move back new op lower bound from 7334324393925402624 to 7334324393790885888. Current Snapshot: MvccSnapshot[applied={T|T < 7334324393812983808}]
W20260928 14:42:21.527212 12467 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:21.525366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45554 (local address 127.22.13.125:33413)
0928 14:42:21.525413 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.525415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.526478 (+ 1063us) server_negotiation.cc:407] Connection header received
0928 14:42:21.526603 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.526604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.526615 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.526647 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.527134 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45554: BlockingRecv error: recv got EOF from 127.0.0.1:45554 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260928 14:42:21.528040 12390 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:21.525814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44556 (local address 127.22.13.126:42527)
0928 14:42:21.525857 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.525858 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.527471 (+ 1613us) server_negotiation.cc:407] Connection header received
0928 14:42:21.527600 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.527601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.527614 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.527643 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.527975 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44556: BlockingRecv error: recv got EOF from 127.0.0.1:44556 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260928 14:42:21.528923 12586 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:21.526189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41608 (local address 127.22.13.124:44037)
0928 14:42:21.526245 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.526246 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.528255 (+ 2009us) server_negotiation.cc:407] Connection header received
0928 14:42:21.528408 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.528409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.528422 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.528465 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.528843 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41608: BlockingRecv error: recv got EOF from 127.0.0.1:41608 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260928 14:42:21.531493 12467 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:21.530691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45564 (local address 127.22.13.125:33413)
0928 14:42:21.530731 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.530733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.530929 (+ 196us) server_negotiation.cc:407] Connection header received
0928 14:42:21.531046 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.531047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.531069 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.531100 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.531444 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45564: BlockingRecv error: recv got EOF from 127.0.0.1:45564 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260928 14:42:21.534749 12467 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:21.533902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45572 (local address 127.22.13.125:33413)
0928 14:42:21.533956 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.533958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.534219 (+ 261us) server_negotiation.cc:407] Connection header received
0928 14:42:21.534319 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.534320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.534331 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.534359 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.534698 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45572: BlockingRecv error: recv got EOF from 127.0.0.1:45572 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260928 14:42:21.538885 12467 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:21.538016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45574 (local address 127.22.13.125:33413)
0928 14:42:21.538056 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.538058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.538335 (+ 277us) server_negotiation.cc:407] Connection header received
0928 14:42:21.538434 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.538435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.538446 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.538476 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.538830 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45574: BlockingRecv error: recv got EOF from 127.0.0.1:45574 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260928 14:42:21.545120 12467 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:21.544253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45576 (local address 127.22.13.125:33413)
0928 14:42:21.544295 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.544297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.544535 (+ 238us) server_negotiation.cc:407] Connection header received
0928 14:42:21.544654 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.544656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.544667 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.544698 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.545060 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45576: BlockingRecv error: recv got EOF from 127.0.0.1:45576 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260928 14:42:21.555665 12467 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:21.554588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45580 (local address 127.22.13.125:33413)
0928 14:42:21.554638 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.554640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.554994 (+ 354us) server_negotiation.cc:407] Connection header received
0928 14:42:21.555129 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.555131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.555143 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.555179 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.555603 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45580: BlockingRecv error: recv got EOF from 127.0.0.1:45580 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260928 14:42:21.574566 12467 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:21.573123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45588 (local address 127.22.13.125:33413)
0928 14:42:21.573174 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.573176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.573577 (+ 401us) server_negotiation.cc:407] Connection header received
0928 14:42:21.573707 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.573708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.573723 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.573757 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.574487 (+ 730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45588: BlockingRecv error: recv got EOF from 127.0.0.1:45588 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260928 14:42:21.609123 12467 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:21.608090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45590 (local address 127.22.13.125:33413)
0928 14:42:21.608140 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.608142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.608471 (+ 329us) server_negotiation.cc:407] Connection header received
0928 14:42:21.608601 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.608602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.608616 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.608648 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.609059 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45590: BlockingRecv error: recv got EOF from 127.0.0.1:45590 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260928 14:42:21.675410 12467 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:21.674596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45604 (local address 127.22.13.125:33413)
0928 14:42:21.674647 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.674650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.674844 (+ 194us) server_negotiation.cc:407] Connection header received
0928 14:42:21.674882 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.674883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.674899 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.674958 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.675344 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45604: BlockingRecv error: recv got EOF from 127.0.0.1:45604 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260928 14:42:21.805620 12467 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:21.804792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45610 (local address 127.22.13.125:33413)
0928 14:42:21.804887 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:21.804902 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:21.804992 (+ 90us) server_negotiation.cc:407] Connection header received
0928 14:42:21.805094 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:21.805095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:21.805107 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:21.805185 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:21.805551 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45610: BlockingRecv error: recv got EOF from 127.0.0.1:45610 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 14:42:21.963567 12574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8096efa5cb664498acea9b2e844a24e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8096efa5cb664498acea9b2e844a24e2" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 42527 } } peers { permanent_uuid: "5e0c2e22ec78420991fcc70c9e872fa1" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 33413 } } peers { permanent_uuid: "ac613f5bcd984d64a5b662872f632695" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 44037 } } }
I20260928 14:42:21.963717 12574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8096efa5cb664498acea9b2e844a24e2 [sys.catalog]: This master's current role is: LEADER
W20260928 14:42:22.063642 12467 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:22.062980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45626 (local address 127.22.13.125:33413)
0928 14:42:22.063070 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:22.063072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:22.063082 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:42:22.063111 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:22.063113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:22.063124 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:22.063203 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:22.063573 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45626: BlockingRecv error: recv got EOF from 127.0.0.1:45626 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 14:42:22.433863 12554 catalog_manager.cc:1259] Loaded cluster ID: 589c8c993a2844e68615260ce51ed438
I20260928 14:42:22.434000 12554 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695: loading cluster ID for follower catalog manager: success
I20260928 14:42:22.434921 12554 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695: acquiring CA information for follower catalog manager: success
I20260928 14:42:22.435369 12554 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ac613f5bcd984d64a5b662872f632695: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:42:22.498009 12600 catalog_manager.cc:1259] Loaded cluster ID: 589c8c993a2844e68615260ce51ed438
I20260928 14:42:22.498147 12600 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1: loading cluster ID for follower catalog manager: success
I20260928 14:42:22.499049 12600 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1: acquiring CA information for follower catalog manager: success
I20260928 14:42:22.499418 12600 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5e0c2e22ec78420991fcc70c9e872fa1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:42:22.579424 12671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:22.576831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45642 (local address 127.22.13.125:33413)
0928 14:42:22.576999 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:22.577001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:22.577012 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:42:22.577040 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:22.577041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:22.577091 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:22.577182 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:22.577340 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:22.577842 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:22.578686 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:22.578890 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:22.578946 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:22.579101 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:22.579227 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:22.579291 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:22.579328 (+ 37us) server_negotiation.cc:299] Negotiation successful
0928 14:42:22.579350 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":63,"threads_started":1}
I20260928 14:42:22.583879 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:22.581435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41622 (local address 127.22.13.124:44037)
0928 14:42:22.581544 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:22.581546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:22.581562 (+ 16us) server_negotiation.cc:407] Connection header received
0928 14:42:22.581602 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:22.581602 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:22.581645 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:22.581722 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:22.581850 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:22.582407 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:22.583138 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:22.583333 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:22.583384 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:22.583522 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:22.583637 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:22.583696 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:22.583733 (+ 37us) server_negotiation.cc:299] Negotiation successful
0928 14:42:22.583751 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":64,"threads_started":1}
I20260928 14:42:22.588689 12671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:22.585697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45654 (local address 127.22.13.125:33413)
0928 14:42:22.585742 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:22.585744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:22.586165 (+ 421us) server_negotiation.cc:407] Connection header received
0928 14:42:22.586229 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:22.586230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:22.586242 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:22.586294 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:22.586396 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:22.587115 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:22.587886 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:22.588191 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:22.588262 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:22.588402 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:22.588441 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:22.588491 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:22.588578 (+ 87us) server_negotiation.cc:299] Negotiation successful
0928 14:42:22.588612 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:42:22.588836 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:22.585775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41628 (local address 127.22.13.124:44037)
0928 14:42:22.586049 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:22.586050 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:22.586059 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:42:22.586085 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:22.586086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:22.586097 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:22.586148 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:22.586333 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:22.586836 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:22.587933 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:22.588230 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:22.588297 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:22.588511 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:22.588546 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:22.588590 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:22.588727 (+ 137us) server_negotiation.cc:299] Negotiation successful
0928 14:42:22.588752 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260928 14:42:22.589133 12683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:22.585704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.22.13.126:42527)
0928 14:42:22.585837 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:22.585839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:22.585847 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:42:22.585916 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:22.585917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:22.586010 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:22.586107 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:22.586236 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:22.587042 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:22.588117 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:22.588416 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:22.588482 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:22.588661 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:22.588779 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:22.588845 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:22.589027 (+ 182us) server_negotiation.cc:299] Negotiation successful
0928 14:42:22.589056 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260928 14:42:22.590863 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 12319
I20260928 14:42:22.596546 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 12394
I20260928 14:42:22.601869 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 12469
2026-09-28T14:42:22Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2786 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:22 dist-test-slave-wwwb krb5kdc[12688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:22 dist-test-slave-wwwb krb5kdc[12688](info): set up 2 sockets
Sep 28 14:42:22 dist-test-slave-wwwb krb5kdc[12688](info): commencing operation
krb5kdc: starting...
W20260928 14:42:23.638473 22581 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/test-user.keytab.
Sep 28 14:42:23 dist-test-slave-wwwb krb5kdc[12688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.13.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
Entry for principal kudu/127.22.13.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab.
I20260928 14:42:23.761128 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:39007
--webserver_interface=127.22.13.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:45011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:39007,127.22.13.125:44395,127.22.13.124:42557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:23.845003 12701 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:42:23.845295 12701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:23.845327 12701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:23.845372 12701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:23.846750 12701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:23.846844 12701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:23.846871 12701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:23.846894 12701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:23.846920 12701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:23.848606 12701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:45011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:39007,127.22.13.125:44395,127.22.13.124:42557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.126.keytab
--principal=kudu/127.22.13.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.126:39007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.13.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:23.848919 12701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:23.849171 12701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:23.854552 12707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:23.854727 12701 server_base.cc:1034] running on GCE node
W20260928 14:42:23.854607 12709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:23.854564 12706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:23.855201 12701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:23.855556 12701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:23.856742 12701 hybrid_clock.cc:648] HybridClock initialized: now 1790606543856713 us; error 48 us; skew 500 ppm
Sep 28 14:42:23 dist-test-slave-wwwb krb5kdc[12688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606543, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:23.865899 12701 init.cc:377] Logged in from keytab as kudu/127.22.13.126@KRBTEST.COM (short username kudu)
I20260928 14:42:23.866430 12701 webserver.cc:460] Webserver started at http://127.22.13.126:36587/ using document root <none> and password file <none>
I20260928 14:42:23.866698 12701 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:23.866772 12701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:23.866921 12701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:23.868068 12701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/data/instance:
uuid: "696a1b1d345e4b94a0df1d69e42a9704"
format_stamp: "Formatted at 2026-09-28 14:42:23 on dist-test-slave-wwwb"
server_key: "db94e23dc74ca6a44daaa76e2da04c42"
server_key_iv: "f1d926f9a7aa38dd97738bd4b65585bb"
server_key_version: "encryptionkey@0"
I20260928 14:42:23.868458 12701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/wal/instance:
uuid: "696a1b1d345e4b94a0df1d69e42a9704"
format_stamp: "Formatted at 2026-09-28 14:42:23 on dist-test-slave-wwwb"
server_key: "db94e23dc74ca6a44daaa76e2da04c42"
server_key_iv: "f1d926f9a7aa38dd97738bd4b65585bb"
server_key_version: "encryptionkey@0"
I20260928 14:42:23.869831 12701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:42:23.870636 12716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:23.870811 12701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:42:23.870929 12701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/wal
uuid: "696a1b1d345e4b94a0df1d69e42a9704"
format_stamp: "Formatted at 2026-09-28 14:42:23 on dist-test-slave-wwwb"
server_key: "db94e23dc74ca6a44daaa76e2da04c42"
server_key_iv: "f1d926f9a7aa38dd97738bd4b65585bb"
server_key_version: "encryptionkey@0"
I20260928 14:42:23.871006 12701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:23.884584 12701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:23.887084 12701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:23.887274 12701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:23.891381 12701 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.126:39007
I20260928 14:42:23.891417 12768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.126:39007 every 8 connection(s)
I20260928 14:42:23.891795 12701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:23.892789 12769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:23.894492 12769 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:42:23.896266 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 12701
I20260928 14:42:23.896373 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-0/wal/instance
I20260928 14:42:23.896559 22581 external_mini_cluster.cc:1428] Setting key f1bec817ed668c8e67808d44078a6668
Sep 28 14:42:23 dist-test-slave-wwwb krb5kdc[12688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606543, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.13.125@KRBTEST.COM; defaulting to no policy
I20260928 14:42:23.904554 12772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:23.895261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47674 (local address 127.22.13.126:39007)
0928 14:42:23.895464 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:23.895475 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:23.895588 (+ 113us) server_negotiation.cc:407] Connection header received
0928 14:42:23.896085 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:23.896087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:23.896105 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:23.896241 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:23.896931 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:23.897760 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:23.898728 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:23.899051 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:23.899205 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:23.901268 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:23.901276 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:23.901277 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:23.901284 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:23.902932 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:23.903331 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:23.903334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:23.903334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:23.903413 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:23.903653 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:23.903654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:23.903655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:23.903974 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:23.904056 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:23.904245 (+ 189us) server_negotiation.cc:299] Negotiation successful
0928 14:42:23.904404 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":166,"thread_start_us":120,"threads_started":1}
I20260928 14:42:23.904583 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:23.895229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39007 (local address 127.0.0.1:47674)
0928 14:42:23.895454 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:23.895495 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:23.895959 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:23.896250 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:23.896252 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:23.896603 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:23.896824 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:23.896829 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:23.897886 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:23.897887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:23.898630 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:23.898633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:23.899187 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:23.899188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:23.899257 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:23.899807 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:23.899816 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:23.901178 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:23.903028 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:23.903034 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:23.903036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:23.903237 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:23.903479 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:23.903480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:23.903481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:23.903569 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:23.904045 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:23.904051 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:23.904113 (+ 62us) client_negotiation.cc:769] Sending connection context
0928 14:42:23.904195 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 14:42:23.904338 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":129,"threads_started":1}
I20260928 14:42:23.905391 12769 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:42:23.905757 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:23.905628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:44395 (local address 127.0.0.1:35822)
0928 14:42:23.905693 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:23.905738 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:44395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260928 14:42:23.905896 12720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.13.125:44395: connect: Connection refused (error 111)
I20260928 14:42:23.906325 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:23.906228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:44395 (local address 127.0.0.1:35836)
0928 14:42:23.906285 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:23.906310 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:44395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.125@KRBTEST.COM" created.
W20260928 14:42:23.906437 12769 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.125:44395: Network error: Client connection negotiation failed: client connection to 127.22.13.125:44395: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.125.keytab.
Entry for principal kudu/127.22.13.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.125.keytab.
I20260928 14:42:23.921038 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.125:44395
--webserver_interface=127.22.13.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.125.keytab
--principal=kudu/127.22.13.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:45011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:39007,127.22.13.125:44395,127.22.13.124:42557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:42:23.955659 12769 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } attempt: 1
I20260928 14:42:23.956192 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:23.956039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:44395 (local address 127.0.0.1:35840)
0928 14:42:23.956125 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:23.956161 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:44395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260928 14:42:23.956535 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:23.956438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:44395 (local address 127.0.0.1:35856)
0928 14:42:23.956485 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:23.956509 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:44395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 14:42:23.956621 12769 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.125:44395: Network error: Client connection negotiation failed: client connection to 127.22.13.125:44395: connect: Connection refused (error 111)
W20260928 14:42:24.003904 12776 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:42:24.004189 12776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:24.004221 12776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:24.004279 12776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:24.005633 12776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:24.005699 12776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:24.005721 12776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:24.005743 12776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:24.005779 12776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:24.007418 12776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:45011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:39007,127.22.13.125:44395,127.22.13.124:42557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.125.keytab
--principal=kudu/127.22.13.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.125:44395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.13.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:24.007735 12776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:24.007977 12776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:24.013149 12784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:24.013154 12782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:24.013144 12781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:24.013294 12776 server_base.cc:1034] running on GCE node
I20260928 14:42:24.013571 12776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:24.013855 12776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:24.014999 12776 hybrid_clock.cc:648] HybridClock initialized: now 1790606544014992 us; error 43 us; skew 500 ppm
Sep 28 14:42:24 dist-test-slave-wwwb krb5kdc[12688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606544, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:24.017470 12776 init.cc:377] Logged in from keytab as kudu/127.22.13.125@KRBTEST.COM (short username kudu)
I20260928 14:42:24.017913 12776 webserver.cc:460] Webserver started at http://127.22.13.125:40825/ using document root <none> and password file <none>
I20260928 14:42:24.018193 12776 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:24.018260 12776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:24.018414 12776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:24.019523 12776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/data/instance:
uuid: "790591f35b0c469487695873f27dba44"
format_stamp: "Formatted at 2026-09-28 14:42:24 on dist-test-slave-wwwb"
server_key: "f7f7edbe019f02c807a4edc953bb0dfd"
server_key_iv: "94a5a16065b469006059d8b25794c798"
server_key_version: "encryptionkey@0"
I20260928 14:42:24.019897 12776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/wal/instance:
uuid: "790591f35b0c469487695873f27dba44"
format_stamp: "Formatted at 2026-09-28 14:42:24 on dist-test-slave-wwwb"
server_key: "f7f7edbe019f02c807a4edc953bb0dfd"
server_key_iv: "94a5a16065b469006059d8b25794c798"
server_key_version: "encryptionkey@0"
I20260928 14:42:24.021282 12776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:42:24.022156 12791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:24.022341 12776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:24.022462 12776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/wal
uuid: "790591f35b0c469487695873f27dba44"
format_stamp: "Formatted at 2026-09-28 14:42:24 on dist-test-slave-wwwb"
server_key: "f7f7edbe019f02c807a4edc953bb0dfd"
server_key_iv: "94a5a16065b469006059d8b25794c798"
server_key_version: "encryptionkey@0"
I20260928 14:42:24.022550 12776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:24.024778 12769 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } attempt: 2
I20260928 14:42:24.025243 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.025104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:44395 (local address 127.0.0.1:35862)
0928 14:42:24.025169 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.025211 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:44395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 14:42:24.025586 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.025514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:44395 (local address 127.0.0.1:35870)
0928 14:42:24.025549 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.025571 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.125:44395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260928 14:42:24.025692 12769 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.125:44395: Network error: Client connection negotiation failed: client connection to 127.22.13.125:44395: connect: Connection refused (error 111)
I20260928 14:42:24.038120 12776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:24.040010 12776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:24.040184 12776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:24.044340 12776 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.125:44395
I20260928 14:42:24.044379 12843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.125:44395 every 8 connection(s)
I20260928 14:42:24.044806 12776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/data/info.pb
I20260928 14:42:24.045814 12844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:24.046222 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 12776
I20260928 14:42:24.046304 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-1/wal/instance
I20260928 14:42:24.046485 22581 external_mini_cluster.cc:1428] Setting key ddddc7942bb528e22d8ec7e3799127d7
I20260928 14:42:24.047726 12844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:42:24 dist-test-slave-wwwb krb5kdc[12688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606544, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.125@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.13.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.13.124@KRBTEST.COM" created.
I20260928 14:42:24.057756 12772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.048664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47678 (local address 127.22.13.126:39007)
0928 14:42:24.048730 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.048733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.048963 (+ 230us) server_negotiation.cc:407] Connection header received
0928 14:42:24.049459 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.049461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.049478 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.049561 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.050358 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.050991 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.052008 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.052221 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.052317 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.054871 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:24.054873 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:24.054874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:24.054883 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:24.056289 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.056755 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.056756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.056757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.056811 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.057075 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.057076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.057076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.057229 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:24.057307 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:24.057607 (+ 300us) server_negotiation.cc:299] Negotiation successful
0928 14:42:24.057685 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:42:24.058110 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.048623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39007 (local address 127.0.0.1:47678)
0928 14:42:24.048824 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.048870 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:24.049360 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:24.049634 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:24.049638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:24.049908 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:24.050258 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:24.050263 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.051079 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.051080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:24.051907 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:24.051909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.052345 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.052346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:24.052480 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.053063 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:24.053072 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:24.054769 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:24.056386 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:24.056397 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:24.056405 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:24.056680 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:24.056864 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:24.056864 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:24.056864 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:24.056993 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:24.057314 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:24.057322 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:24.057464 (+ 142us) client_negotiation.cc:769] Sending connection context
0928 14:42:24.057583 (+ 119us) client_negotiation.cc:240] Negotiation successful
0928 14:42:24.057780 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":119,"threads_started":1}
I20260928 14:42:24.058763 12844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:42:24 dist-test-slave-wwwb krb5kdc[12688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606544, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.125@KRBTEST.COM for kudu/127.22.13.125@KRBTEST.COM
I20260928 14:42:24.066386 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.059053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:44395 (local address 127.0.0.1:35886)
0928 14:42:24.059242 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.059253 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:24.059313 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:24.059462 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:24.059463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:24.059603 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:24.059655 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:24.059656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.060513 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.060514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:24.061034 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:24.061036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.061534 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.061535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:24.061599 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.062075 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:24.062081 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:24.063267 (+ 1186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:24.065332 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:24.065333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:24.065334 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:24.065501 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:24.065700 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:24.065701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:24.065701 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:24.065752 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:24.066098 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:24.066099 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:24.066148 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 14:42:24.066215 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 14:42:24.066313 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260928 14:42:24.066506 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.059101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35886 (local address 127.22.13.125:44395)
0928 14:42:24.059256 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.059260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.059313 (+ 53us) server_negotiation.cc:407] Connection header received
0928 14:42:24.059371 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.059373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.059386 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.059461 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.059736 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.060431 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.061110 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.061446 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.061534 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.063359 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:24.063369 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:24.063369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:24.063377 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:24.065242 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.065580 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.065583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.065583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.065641 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.065811 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.065811 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.065812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.066025 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:24.066114 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:24.066250 (+ 136us) server_negotiation.cc:299] Negotiation successful
0928 14:42:24.066366 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260928 14:42:24.067099 12844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:42:24.067524 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.067387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:42557 (local address 127.0.0.1:43012)
0928 14:42:24.067442 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.067496 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 14:42:24.067679 12794 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.13.124:42557: connect: Connection refused (error 111)
I20260928 14:42:24.068081 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.067988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:42557 (local address 127.0.0.1:43028)
0928 14:42:24.068034 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.068059 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 14:42:24.068183 12844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.124:42557: Network error: Client connection negotiation failed: client connection to 127.22.13.124:42557: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.13.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.124.keytab.
Entry for principal kudu/127.22.13.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.124.keytab.
I20260928 14:42:24.073150 22581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
/tmp/dist-test-taskT1ZEJk/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.124:42557
--webserver_interface=127.22.13.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.124.keytab
--principal=kudu/127.22.13.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.13.84:45011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.13.126:39007,127.22.13.125:44395,127.22.13.124:42557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:42:24.116866 12769 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } attempt: 3
I20260928 14:42:24.117380 12844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } attempt: 1
I20260928 14:42:24.117774 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.117636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:42557 (local address 127.0.0.1:43036)
0928 14:42:24.117699 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.117733 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260928 14:42:24.118134 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.118036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:42557 (local address 127.0.0.1:43052)
0928 14:42:24.118082 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.118110 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 14:42:24.118227 12844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.124:42557: Network error: Client connection negotiation failed: client connection to 127.22.13.124:42557: connect: Connection refused (error 111)
Sep 28 14:42:24 dist-test-slave-wwwb krb5kdc[12688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606543, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for kudu/127.22.13.125@KRBTEST.COM
I20260928 14:42:24.124693 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.117379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35900 (local address 127.22.13.125:44395)
0928 14:42:24.117422 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.117424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.117434 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:42:24.117582 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.117583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.117597 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.117656 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.118124 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.118738 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.119739 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.119952 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.120000 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.122053 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:24.122054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:24.122054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:24.122063 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:24.123527 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.123893 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.123895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.123895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.123955 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.124151 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.124153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.124153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.124278 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:24.124351 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:24.124503 (+ 152us) server_negotiation.cc:299] Negotiation successful
0928 14:42:24.124590 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:42:24.124693 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.117290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:44395 (local address 127.0.0.1:35900)
0928 14:42:24.117365 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.117380 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:24.117453 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:24.117711 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:24.117712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:24.117912 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:24.118021 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:24.118024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.118829 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.118830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:24.119651 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:24.119654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.120019 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.120020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:24.120099 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.120531 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:24.120537 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:24.121912 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:24.123632 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:24.123633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:24.123634 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:24.123821 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:24.124022 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:24.124023 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:24.124023 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:24.124088 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:24.124343 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:24.124344 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:24.124397 (+ 53us) client_negotiation.cc:769] Sending connection context
0928 14:42:24.124479 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 14:42:24.124598 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260928 14:42:24.125170 12769 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:42:24.125574 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.125434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:42557 (local address 127.0.0.1:43054)
0928 14:42:24.125491 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.125529 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260928 14:42:24.125994 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.125871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:42557 (local address 127.0.0.1:43058)
0928 14:42:24.125918 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.125969 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 14:42:24.126076 12769 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.124:42557: Network error: Client connection negotiation failed: client connection to 127.22.13.124:42557: connect: Connection refused (error 111)
I20260928 14:42:24.157262 12769 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } attempt: 1
I20260928 14:42:24.157826 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.157644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:42557 (local address 127.0.0.1:43072)
0928 14:42:24.157715 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.157763 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260928 14:42:24.158221 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.158120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:42557 (local address 127.0.0.1:43076)
0928 14:42:24.158169 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.158198 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 14:42:24.158351 12769 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.124:42557: Network error: Client connection negotiation failed: client connection to 127.22.13.124:42557: connect: Connection refused (error 111)
W20260928 14:42:24.159539 12851 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:42:24.159765 12851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:24.159825 12851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:24.159869 12851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:24.161046 12851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:24.161088 12851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:24.161103 12851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:24.161114 12851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:24.161157 12851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:24.162770 12851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.13.84:45011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.13.126:39007,127.22.13.125:44395,127.22.13.124:42557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/krb5kdc/kudu_127.22.13.124.keytab
--principal=kudu/127.22.13.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.13.124:42557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.13.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:24.163033 12851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:24.163286 12851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:24.168267 12857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:24.168327 12851 server_base.cc:1034] running on GCE node
W20260928 14:42:24.168263 12856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:24.168263 12859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:24.168704 12851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:24.169057 12851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:24.170202 12851 hybrid_clock.cc:648] HybridClock initialized: now 1790606544170201 us; error 40 us; skew 500 ppm
Sep 28 14:42:24 dist-test-slave-wwwb krb5kdc[12688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606544, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:24.172731 12851 init.cc:377] Logged in from keytab as kudu/127.22.13.124@KRBTEST.COM (short username kudu)
I20260928 14:42:24.173254 12851 webserver.cc:460] Webserver started at http://127.22.13.124:40417/ using document root <none> and password file <none>
I20260928 14:42:24.173506 12851 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:24.173574 12851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:24.173719 12851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:24.174961 12851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/data/instance:
uuid: "ae97547f5f034916b7ae77b496291be4"
format_stamp: "Formatted at 2026-09-28 14:42:24 on dist-test-slave-wwwb"
server_key: "f50f821b6e32b29aeebe592128aaae23"
server_key_iv: "6511e55eb7a9d10ddc7818da27590115"
server_key_version: "encryptionkey@0"
I20260928 14:42:24.175356 12851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/wal/instance:
uuid: "ae97547f5f034916b7ae77b496291be4"
format_stamp: "Formatted at 2026-09-28 14:42:24 on dist-test-slave-wwwb"
server_key: "f50f821b6e32b29aeebe592128aaae23"
server_key_iv: "6511e55eb7a9d10ddc7818da27590115"
server_key_version: "encryptionkey@0"
I20260928 14:42:24.176798 12851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:42:24.177536 12866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:24.177718 12851 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:42:24.177822 12851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/data,/tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/wal
uuid: "ae97547f5f034916b7ae77b496291be4"
format_stamp: "Formatted at 2026-09-28 14:42:24 on dist-test-slave-wwwb"
server_key: "f50f821b6e32b29aeebe592128aaae23"
server_key_iv: "6511e55eb7a9d10ddc7818da27590115"
server_key_version: "encryptionkey@0"
I20260928 14:42:24.177899 12851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:24.186412 12851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:24.186388 12844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } attempt: 2
I20260928 14:42:24.186879 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.186753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:42557 (local address 127.0.0.1:43080)
0928 14:42:24.186809 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.186846 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 14:42:24.187299 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.187170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:42557 (local address 127.0.0.1:43084)
0928 14:42:24.187215 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.187259 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.13.124:42557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260928 14:42:24.187390 12844 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.13.124:42557: Network error: Client connection negotiation failed: client connection to 127.22.13.124:42557: connect: Connection refused (error 111)
I20260928 14:42:24.189075 12851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:24.189253 12851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:24.193216 12851 rpc_server.cc:307] RPC server started. Bound to: 127.22.13.124:42557
I20260928 14:42:24.193251 12918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.13.124:42557 every 8 connection(s)
I20260928 14:42:24.193657 12851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/data/info.pb
I20260928 14:42:24.194741 12919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:24.196466 12919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:42:24.198421 22581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu as pid 12851
I20260928 14:42:24.198529 22581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskT1ZEJk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606270465379-22581-0/minicluster-data/master-2/wal/instance
I20260928 14:42:24.198747 22581 external_mini_cluster.cc:1428] Setting key df25a831441898b0c494730b02808409
Sep 28 14:42:24 dist-test-slave-wwwb krb5kdc[12688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606544, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.124@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
Sep 28 14:42:24 dist-test-slave-wwwb krb5kdc[12688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.124@KRBTEST.COM
I20260928 14:42:24.206611 12772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.197232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47694 (local address 127.22.13.126:39007)
0928 14:42:24.197305 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.197308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.197565 (+ 257us) server_negotiation.cc:407] Connection header received
0928 14:42:24.198125 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.198127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.198137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.198192 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.199130 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.199835 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.200968 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.201220 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.201272 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.203660 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:24.203662 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:24.203662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:24.203672 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:24.205131 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.205561 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.205562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.205563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.205621 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.205901 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.205902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.205902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.206056 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:24.206129 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:24.206401 (+ 272us) server_negotiation.cc:299] Negotiation successful
0928 14:42:24.206487 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 14:42:24.206856 12921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.197221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.126:39007 (local address 127.0.0.1:47694)
0928 14:42:24.197436 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.197479 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:24.198033 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:24.198300 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:24.198305 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:24.198711 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:24.199012 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:24.199018 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.199928 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.199929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:24.200883 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:24.200885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.201318 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.201320 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:24.201468 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.202051 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:24.202061 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:24.203569 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:24.205232 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:24.205241 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:24.205244 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:24.205486 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:24.205684 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:24.205685 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:24.205685 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:24.205808 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:24.206121 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:24.206128 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:24.206268 (+ 140us) client_negotiation.cc:769] Sending connection context
0928 14:42:24.206380 (+ 112us) client_negotiation.cc:240] Negotiation successful
0928 14:42:24.206604 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":113,"threads_started":1}
I20260928 14:42:24.207643 12919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:42:24.208093 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.199187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.22.13.124:42557)
0928 14:42:24.199365 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.199370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.199455 (+ 85us) server_negotiation.cc:407] Connection header received
0928 14:42:24.199478 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.199480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.199502 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.199582 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.200275 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.201026 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.202037 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.202479 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.202555 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.205024 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:24.205032 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:24.205032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:24.205040 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:24.206650 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.207161 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.207164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.207165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.207219 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.207426 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.207427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.207427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.207567 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:24.207713 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:24.207808 (+ 95us) server_negotiation.cc:299] Negotiation successful
0928 14:42:24.207887 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
Sep 28 14:42:24 dist-test-slave-wwwb krb5kdc[12688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606544, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.124@KRBTEST.COM for kudu/127.22.13.125@KRBTEST.COM
Sep 28 14:42:24 dist-test-slave-wwwb krb5kdc[12688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.126@KRBTEST.COM
I20260928 14:42:24.214942 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.207993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35906 (local address 127.22.13.125:44395)
0928 14:42:24.208039 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.208041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.208053 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:42:24.208079 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.208080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.208102 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.208168 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.208468 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.209150 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.209894 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.210155 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.210208 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.212166 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:24.212167 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:24.212168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:24.212176 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:24.213744 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.214169 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.214170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.214170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.214219 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.214423 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.214424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.214424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.214537 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:24.214613 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:24.214763 (+ 150us) server_negotiation.cc:299] Negotiation successful
0928 14:42:24.214845 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:42:24.214942 12921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.207926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.125:44395 (local address 127.0.0.1:35906)
0928 14:42:24.207970 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.207981 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:24.208019 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:24.208181 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:24.208182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:24.208337 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:24.208398 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:24.208400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.209233 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.209234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:24.209816 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:24.209818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.210221 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.210222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:24.210296 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.210706 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:24.210711 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:24.212075 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:24.213824 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:24.213825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:24.213826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:24.214079 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:24.214292 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:24.214293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:24.214294 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:24.214360 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:24.214605 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:24.214606 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:24.214656 (+ 50us) client_negotiation.cc:769] Sending connection context
0928 14:42:24.214737 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 14:42:24.214845 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260928 14:42:24.215392 12919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:42:24.217072 12772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.208876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47710 (local address 127.22.13.126:39007)
0928 14:42:24.209253 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.209254 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.209262 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:42:24.209289 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.209290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.209300 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.209350 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.209818 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.210418 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.211204 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.211387 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.211438 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.214241 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:24.214242 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:24.214242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:24.214249 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:24.215917 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.216269 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.216270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.216271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.216328 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.216521 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.216522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.216522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.216650 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:24.216852 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:24.216889 (+ 37us) server_negotiation.cc:299] Negotiation successful
0928 14:42:24.216966 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355}
Sep 28 14:42:24 dist-test-slave-wwwb krb5kdc[12688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606544, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.124@KRBTEST.COM for kudu/127.22.13.124@KRBTEST.COM
Sep 28 14:42:24 dist-test-slave-wwwb krb5kdc[12688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.13.125@KRBTEST.COM
I20260928 14:42:24.222662 12921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.215671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:42557 (local address 127.0.0.1:43102)
0928 14:42:24.215717 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.215728 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:24.215772 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:24.215906 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:24.215907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:24.216040 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:24.216070 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:24.216071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.216700 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.216701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:24.217337 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:24.217339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.217899 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.217899 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:24.218007 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.218411 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:24.218416 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:24.219764 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:24.221565 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:24.221566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:24.221567 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:24.221781 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:24.222015 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:24.222016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:24.222016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:24.222076 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:24.222357 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:24.222358 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:24.222395 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 14:42:24.222446 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 14:42:24.222520 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260928 14:42:24.222735 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.215710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43102 (local address 127.22.13.124:42557)
0928 14:42:24.215745 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.215747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.215758 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:42:24.215835 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.215835 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.215846 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.215903 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.216137 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.216626 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.217407 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.217659 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.217862 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.219858 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:24.219859 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:24.219859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:24.219866 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:24.221460 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.221867 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.221868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.221868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.221955 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.222144 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.222145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.222145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.222291 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:24.222362 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:24.222472 (+ 110us) server_negotiation.cc:299] Negotiation successful
0928 14:42:24.222590 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":20}
I20260928 14:42:24.224362 12919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4: Bootstrap starting.
I20260928 14:42:24.225453 12919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:24.225711 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.217683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35918 (local address 127.22.13.125:44395)
0928 14:42:24.217728 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.217730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.217737 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:42:24.217757 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.217758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.217767 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.217817 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.218441 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.219089 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.219886 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.220169 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.220236 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.222994 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:24.222996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:24.222996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:24.223006 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:24.224483 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.224905 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.224906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.224906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.224956 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.225138 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.225138 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.225138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.225236 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:24.225363 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:24.225499 (+ 136us) server_negotiation.cc:299] Negotiation successful
0928 14:42:24.225580 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:42:24.225766 12919 log.cc:824] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:24.226862 12919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4: No bootstrap required, opened a new log
I20260928 14:42:24.227571 22581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:24.228817 12919 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } }
I20260928 14:42:24.228948 12919 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae97547f5f034916b7ae77b496291be4, State: Initialized, Role: FOLLOWER
I20260928 14:42:24.229035 12919 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } }
I20260928 14:42:24.228521 12743 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "696a1b1d345e4b94a0df1d69e42a9704"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47710
I20260928 14:42:24.229375 12919 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:24.229358 12925 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } } }
I20260928 14:42:24.229517 12925 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4 [sys.catalog]: This master's current role is: FOLLOWER
W20260928 14:42:24.231175 12936 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:42:24.231542 12936 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:42:24.233481 12769 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } attempt: 2
I20260928 14:42:24.231595 12851 master_runner.cc:186] Error getting master registration for 127.22.13.126:39007: OK, instance_id { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" instance_seqno: 1790606543890158 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:42:24.234639 12851 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 28 14:42:24 dist-test-slave-wwwb krb5kdc[12688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606543, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.126@KRBTEST.COM for kudu/127.22.13.124@KRBTEST.COM
I20260928 14:42:24.241006 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.233829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:42557 (local address 127.0.0.1:43114)
0928 14:42:24.233884 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.233897 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:24.233983 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:24.234149 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:24.234150 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:24.234349 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:24.234405 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:24.234408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.235110 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.235111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:24.235922 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:24.235925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.236284 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.236285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:24.236343 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.236759 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:24.236780 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:24.238155 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:24.239912 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:24.239914 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:24.239915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:24.240111 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:24.240319 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:24.240320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:24.240321 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:24.240382 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:24.240667 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:24.240668 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:24.240716 (+ 48us) client_negotiation.cc:769] Sending connection context
0928 14:42:24.240800 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 14:42:24.240905 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260928 14:42:24.241006 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.233876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43114 (local address 127.22.13.124:42557)
0928 14:42:24.233923 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.233925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.233958 (+ 33us) server_negotiation.cc:407] Connection header received
0928 14:42:24.234046 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.234063 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.234074 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.234143 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.234493 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.235045 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.236016 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.236218 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.236266 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.238250 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:24.238251 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:24.238252 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:24.238258 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:24.239836 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.240192 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.240193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.240194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.240254 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.240443 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.240444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.240445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.240596 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:24.240674 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:24.240824 (+ 150us) server_negotiation.cc:299] Negotiation successful
0928 14:42:24.240906 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:42:24.242858 12769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704: Bootstrap starting.
I20260928 14:42:24.243491 12769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:24.243784 12769 log.cc:824] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:24.244520 12769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704: No bootstrap required, opened a new log
I20260928 14:42:24.245563 12769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } }
I20260928 14:42:24.245712 12769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696a1b1d345e4b94a0df1d69e42a9704, State: Initialized, Role: FOLLOWER
I20260928 14:42:24.245833 12769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } }
I20260928 14:42:24.246133 12939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } } }
I20260928 14:42:24.246191 12769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:24.246230 12743 raft_consensus.cc:482] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260928 14:42:24.246331 12743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:24.247162 12743 raft_consensus.cc:504] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } }
W20260928 14:42:24.248106 12950 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260928 14:42:24.248121 12893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "696a1b1d345e4b94a0df1d69e42a9704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ae97547f5f034916b7ae77b496291be4"
I20260928 14:42:24.248265 12893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4 [term 0 FOLLOWER]: Advancing to term 1
W20260928 14:42:24.248224 12950 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:42:24.249162 12893 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 696a1b1d345e4b94a0df1d69e42a9704 in term 1.
I20260928 14:42:24.249460 12719 leader_election.cc:302] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 696a1b1d345e4b94a0df1d69e42a9704, ae97547f5f034916b7ae77b496291be4; no voters:
I20260928 14:42:24.249665 12743 leader_election.cc:288] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [CANDIDATE]: Term 1 election: Requested vote from peers 790591f35b0c469487695873f27dba44 (127.22.13.125:44395), ae97547f5f034916b7ae77b496291be4 (127.22.13.124:42557)
I20260928 14:42:24.246245 12939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:42:24.250810 12939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:24.250942 12939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [term 1 LEADER]: Becoming Leader. State: Replica: 696a1b1d345e4b94a0df1d69e42a9704, State: Running, Role: LEADER
I20260928 14:42:24.251084 12939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } }
I20260928 14:42:24.249053 12701 master_runner.cc:186] Error getting master registration for 127.22.13.125:44395: OK, instance_id { permanent_uuid: "790591f35b0c469487695873f27dba44" instance_seqno: 1790606544043154 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:42:24.251549 12701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:42:24.251838 12952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 696a1b1d345e4b94a0df1d69e42a9704. Latest consensus state: current_term: 1 leader_uuid: "696a1b1d345e4b94a0df1d69e42a9704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } } }
I20260928 14:42:24.252012 12952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:24.253562 12958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:24.254568 12958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:24.264842 12849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.252388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35926 (local address 127.22.13.125:44395)
0928 14:42:24.252439 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.252441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.253883 (+ 1442us) server_negotiation.cc:407] Connection header received
0928 14:42:24.254013 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.254014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.254024 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.254055 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.256120 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.256841 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.257889 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.258223 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.258265 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.259921 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:24.259922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:24.259922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:24.259930 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:24.261597 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.263488 (+ 1891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.263489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.263490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.263552 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.263747 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.263748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.263748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.263881 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:24.263950 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:24.264428 (+ 478us) server_negotiation.cc:299] Negotiation successful
0928 14:42:24.264520 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:42:24.266481 12893 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4 [term 1 FOLLOWER]: Refusing update from remote peer 696a1b1d345e4b94a0df1d69e42a9704: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:24.266948 12952 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:24.267284 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.252630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43122 (local address 127.22.13.124:42557)
0928 14:42:24.252682 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.252684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.253394 (+ 710us) server_negotiation.cc:407] Connection header received
0928 14:42:24.253674 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.253675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.253687 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.253718 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.256619 (+ 2901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.257337 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.260996 (+ 3659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.261300 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.261346 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.263865 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:24.263867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:24.263868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:24.263875 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:24.265453 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.266095 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.266108 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.266108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.266177 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.266487 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.266488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.266488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.266650 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:24.266734 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:24.267066 (+ 332us) server_negotiation.cc:299] Negotiation successful
0928 14:42:24.267175 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 14:42:24.268936 12925 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 696a1b1d345e4b94a0df1d69e42a9704. Latest consensus state: current_term: 1 leader_uuid: "696a1b1d345e4b94a0df1d69e42a9704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } } }
I20260928 14:42:24.269039 12925 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:42:24.269374 12772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.252047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47722 (local address 127.22.13.126:39007)
0928 14:42:24.252091 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.252093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.252863 (+ 770us) server_negotiation.cc:407] Connection header received
0928 14:42:24.252966 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.252967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.252978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.253008 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.254768 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.255542 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.260071 (+ 4529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.260375 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.260419 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.265601 (+ 5182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:24.265603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:24.265603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:24.265612 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:24.267739 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.268290 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.268302 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.268303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.268358 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.268645 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.268646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.268646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.268789 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:24.268882 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:24.269179 (+ 297us) server_negotiation.cc:299] Negotiation successful
0928 14:42:24.269274 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:42:24.275557 12952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "696a1b1d345e4b94a0df1d69e42a9704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } } }
I20260928 14:42:24.275666 12952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:24.276257 12958 catalog_manager.cc:1347] Generated new cluster ID: d9df25dded984df3abd50a20eba71fb9
I20260928 14:42:24.276312 12958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:42:24.276261 12925 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "696a1b1d345e4b94a0df1d69e42a9704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } } }
I20260928 14:42:24.276360 12925 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:42:24.276896 12963 mvcc.cc:205] Tried to move back new op lower bound from 7334324405311909888 to 7334324405253992448. Current Snapshot: MvccSnapshot[applied={T|T < 7334324405311909888}]
I20260928 14:42:24.278529 12844 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } attempt: 3
Sep 28 14:42:24 dist-test-slave-wwwb krb5kdc[12688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606544, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.13.125@KRBTEST.COM for kudu/127.22.13.124@KRBTEST.COM
I20260928 14:42:24.287192 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.278820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.13.124:42557 (local address 127.0.0.1:43138)
0928 14:42:24.278951 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:24.278963 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:24.279006 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:24.279264 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:24.279265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:24.279466 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:24.279517 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:24.279519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.280485 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.280486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:24.281329 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:24.281331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.281890 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.281891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:24.282005 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.282511 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:24.282517 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:24.284141 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:24.286100 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:24.286101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:24.286101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:24.286320 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:24.286531 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:24.286532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:24.286532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:24.286589 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:24.286867 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:24.286868 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:24.286915 (+ 47us) client_negotiation.cc:769] Sending connection context
0928 14:42:24.286978 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 14:42:24.287083 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260928 14:42:24.287734 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:24.279089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43138 (local address 127.22.13.124:42557)
0928 14:42:24.279135 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.279137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.279143 (+ 6us) server_negotiation.cc:407] Connection header received
0928 14:42:24.279164 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.279164 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.279173 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.279218 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.279599 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.280375 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.281443 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:24.281757 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:24.281816 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:24.284245 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:24.284246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:24.284247 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:24.284254 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:24.286002 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.286406 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.286407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.286408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.286465 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:24.286651 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:24.286651 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:24.286652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:24.286783 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:24.286842 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:24.287548 (+ 706us) server_negotiation.cc:299] Negotiation successful
0928 14:42:24.287630 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260928 14:42:24.289659 12844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44: Bootstrap starting.
I20260928 14:42:24.290499 12844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:24.290814 12844 log.cc:824] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:24.294700 12844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44: No bootstrap required, opened a new log
I20260928 14:42:24.296895 12844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } }
I20260928 14:42:24.297048 12844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 790591f35b0c469487695873f27dba44, State: Initialized, Role: FOLLOWER
I20260928 14:42:24.297137 12844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } }
I20260928 14:42:24.297493 12844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:24.297600 12818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "696a1b1d345e4b94a0df1d69e42a9704" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "790591f35b0c469487695873f27dba44"
I20260928 14:42:24.297731 12818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:24.298698 12818 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 696a1b1d345e4b94a0df1d69e42a9704 in term 1.
I20260928 14:42:24.299175 12817 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44 [term 1 FOLLOWER]: Refusing update from remote peer 696a1b1d345e4b94a0df1d69e42a9704: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:24.299327 12968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } } }
I20260928 14:42:24.299638 12952 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [LEADER]: Connected to new peer: Peer: permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:24.299690 12968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:42:24.299780 12958 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:24.300529 12958 catalog_manager.cc:1504] Loading token signing keys...
W20260928 14:42:24.300922 12979 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:42:24.301033 12979 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:42:24.302361 12968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 696a1b1d345e4b94a0df1d69e42a9704. Latest consensus state: current_term: 1 leader_uuid: "696a1b1d345e4b94a0df1d69e42a9704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } } }
I20260928 14:42:24.302428 12968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:42:24.302455 12968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "696a1b1d345e4b94a0df1d69e42a9704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } } }
I20260928 14:42:24.302487 12968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:42:24.306830 12939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "696a1b1d345e4b94a0df1d69e42a9704" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696a1b1d345e4b94a0df1d69e42a9704" member_type: VOTER last_known_addr { host: "127.22.13.126" port: 39007 } } peers { permanent_uuid: "790591f35b0c469487695873f27dba44" member_type: VOTER last_known_addr { host: "127.22.13.125" port: 44395 } } peers { permanent_uuid: "ae97547f5f034916b7ae77b496291be4" member_type: VOTER last_known_addr { host: "127.22.13.124" port: 42557 } } }
I20260928 14:42:24.306954 12939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:24.307255 12958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 696a1b1d345e4b94a0df1d69e42a9704: Generated new TSK 0
I20260928 14:42:24.307435 12958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:24.318565 12981 mvcc.cc:205] Tried to move back new op lower bound from 7334324405436260352 to 7334324405253992448. Current Snapshot: MvccSnapshot[applied={T|T < 7334324405311909888}]
W20260928 14:42:24.333012 12772 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:24.331496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47724 (local address 127.22.13.126:39007)
0928 14:42:24.331535 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.331537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.332270 (+ 733us) server_negotiation.cc:407] Connection header received
0928 14:42:24.332385 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.332387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.332399 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.332432 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.332935 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47724: BlockingRecv error: recv got EOF from 127.0.0.1:47724 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260928 14:42:24.334151 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:24.331855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35934 (local address 127.22.13.125:44395)
0928 14:42:24.331898 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.331900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.333371 (+ 1471us) server_negotiation.cc:407] Connection header received
0928 14:42:24.333657 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.333658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.333669 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.333695 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.334087 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35934: BlockingRecv error: recv got EOF from 127.0.0.1:35934 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260928 14:42:24.334633 12923 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:24.331283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43146 (local address 127.22.13.124:42557)
0928 14:42:24.331329 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.331331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.333301 (+ 1970us) server_negotiation.cc:407] Connection header received
0928 14:42:24.333545 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.333546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.333556 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.333584 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.334561 (+ 977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43146: BlockingRecv error: recv got EOF from 127.0.0.1:43146 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260928 14:42:24.337064 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:24.336277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35936 (local address 127.22.13.125:44395)
0928 14:42:24.336315 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.336317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.336543 (+ 226us) server_negotiation.cc:407] Connection header received
0928 14:42:24.336670 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.336671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.336691 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.336720 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.337018 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35936: BlockingRecv error: recv got EOF from 127.0.0.1:35936 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260928 14:42:24.340085 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:24.339274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35942 (local address 127.22.13.125:44395)
0928 14:42:24.339308 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.339310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.339560 (+ 250us) server_negotiation.cc:407] Connection header received
0928 14:42:24.339670 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.339671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.339681 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.339709 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.340038 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35942: BlockingRecv error: recv got EOF from 127.0.0.1:35942 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260928 14:42:24.343950 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:24.343133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35948 (local address 127.22.13.125:44395)
0928 14:42:24.343168 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.343170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.343415 (+ 245us) server_negotiation.cc:407] Connection header received
0928 14:42:24.343513 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.343514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.343525 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.343553 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.343901 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35948: BlockingRecv error: recv got EOF from 127.0.0.1:35948 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260928 14:42:24.350035 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:24.349121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35960 (local address 127.22.13.125:44395)
0928 14:42:24.349164 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.349166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.349422 (+ 256us) server_negotiation.cc:407] Connection header received
0928 14:42:24.349518 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.349519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.349529 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.349558 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.349980 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35960: BlockingRecv error: recv got EOF from 127.0.0.1:35960 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260928 14:42:24.360286 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:24.359265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35964 (local address 127.22.13.125:44395)
0928 14:42:24.359312 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.359314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.359614 (+ 300us) server_negotiation.cc:407] Connection header received
0928 14:42:24.359731 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.359732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.359744 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.359776 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.360208 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35964: BlockingRecv error: recv got EOF from 127.0.0.1:35964 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260928 14:42:24.378901 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:24.377879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35970 (local address 127.22.13.125:44395)
0928 14:42:24.377926 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.377928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.378241 (+ 313us) server_negotiation.cc:407] Connection header received
0928 14:42:24.378355 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.378356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.378369 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.378400 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.378840 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35970: BlockingRecv error: recv got EOF from 127.0.0.1:35970 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260928 14:42:24.413236 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:24.412298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35982 (local address 127.22.13.125:44395)
0928 14:42:24.412346 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.412348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.412636 (+ 288us) server_negotiation.cc:407] Connection header received
0928 14:42:24.412738 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.412739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.412752 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.412794 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.413174 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35982: BlockingRecv error: recv got EOF from 127.0.0.1:35982 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260928 14:42:24.479348 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:24.478592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35992 (local address 127.22.13.125:44395)
0928 14:42:24.478666 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.478668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.478784 (+ 116us) server_negotiation.cc:407] Connection header received
0928 14:42:24.478860 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.478861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.478874 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.478934 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.479279 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35992: BlockingRecv error: recv got EOF from 127.0.0.1:35992 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260928 14:42:24.609272 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:24.608560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36002 (local address 127.22.13.125:44395)
0928 14:42:24.608661 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.608663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.608694 (+ 31us) server_negotiation.cc:407] Connection header received
0928 14:42:24.608774 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.608775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.608787 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.608852 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.609204 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36002: BlockingRecv error: recv got EOF from 127.0.0.1:36002 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260928 14:42:24.867270 12849 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:24.866592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36008 (local address 127.22.13.125:44395)
0928 14:42:24.866673 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:24.866675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:24.866729 (+ 54us) server_negotiation.cc:407] Connection header received
0928 14:42:24.866782 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:24.866783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:24.866794 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:24.866847 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:24.867200 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36008: BlockingRecv error: recv got EOF from 127.0.0.1:36008 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:42:25.232023 12936 catalog_manager.cc:1259] Loaded cluster ID: d9df25dded984df3abd50a20eba71fb9
I20260928 14:42:25.232165 12936 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4: loading cluster ID for follower catalog manager: success
I20260928 14:42:25.232995 12936 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4: acquiring CA information for follower catalog manager: success
I20260928 14:42:25.233335 12936 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ae97547f5f034916b7ae77b496291be4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:42:25.301538 12979 catalog_manager.cc:1259] Loaded cluster ID: d9df25dded984df3abd50a20eba71fb9
I20260928 14:42:25.301649 12979 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44: loading cluster ID for follower catalog manager: success
I20260928 14:42:25.302502 12979 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44: acquiring CA information for follower catalog manager: success
I20260928 14:42:25.302847 12979 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 790591f35b0c469487695873f27dba44: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:42:25.383172 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:25.380447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36010 (local address 127.22.13.125:44395)
0928 14:42:25.380558 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:25.380560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:25.380673 (+ 113us) server_negotiation.cc:407] Connection header received
0928 14:42:25.380725 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:25.380726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:25.380781 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:25.380856 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:25.381024 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:25.381580 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:25.382413 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:25.382616 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:25.382672 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:25.382834 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:25.382951 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:25.383011 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:25.383043 (+ 32us) server_negotiation.cc:299] Negotiation successful
0928 14:42:25.383061 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":59,"threads_started":1}
I20260928 14:42:25.387457 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:25.385028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43152 (local address 127.22.13.124:42557)
0928 14:42:25.385139 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:25.385141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:25.385150 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:42:25.385220 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:25.385220 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:25.385259 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:25.385332 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:25.385414 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:25.385909 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:25.386676 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:25.386873 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:25.386925 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:25.387053 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:25.387184 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:25.387246 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:25.387322 (+ 76us) server_negotiation.cc:299] Negotiation successful
0928 14:42:25.387342 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73,"thread_start_us":49,"threads_started":1}
I20260928 14:42:25.392369 13051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:25.389435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36014 (local address 127.22.13.125:44395)
0928 14:42:25.389482 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:25.389483 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:25.389564 (+ 81us) server_negotiation.cc:407] Connection header received
0928 14:42:25.389633 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:25.389634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:25.389646 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:25.389696 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:25.389864 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:25.390409 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:25.391515 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:25.391815 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:25.391874 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:25.392053 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:25.392091 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:25.392137 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:25.392239 (+ 102us) server_negotiation.cc:299] Negotiation successful
0928 14:42:25.392268 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:42:25.392416 13062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:25.389099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47730 (local address 127.22.13.126:39007)
0928 14:42:25.389263 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:25.389266 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:25.389392 (+ 126us) server_negotiation.cc:407] Connection header received
0928 14:42:25.389550 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:25.389551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:25.389607 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:25.389700 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:25.389844 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:25.390600 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:25.391424 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:25.391726 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:25.391793 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:25.392031 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:25.392156 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:25.392195 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:25.392317 (+ 122us) server_negotiation.cc:299] Negotiation successful
0928 14:42:25.392340 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":65,"threads_started":1}
I20260928 14:42:25.392400 13057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:25.389174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43166 (local address 127.22.13.124:42557)
0928 14:42:25.389220 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:25.389222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:25.389427 (+ 205us) server_negotiation.cc:407] Connection header received
0928 14:42:25.389535 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:25.389536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:25.389546 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:25.389595 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:25.389768 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:25.390559 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:25.391633 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:25.391913 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:25.391985 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:25.392106 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:25.392144 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:25.392199 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:25.392297 (+ 98us) server_negotiation.cc:299] Negotiation successful
0928 14:42:25.392325 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:42:25.394313 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 12701
I20260928 14:42:25.400182 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 12776
I20260928 14:42:25.405720 22581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskT1ZEJk/build/release/bin/kudu with pid 12851
2026-09-28T14:42:25Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2803 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5590 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (274947 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST